Author: Robert Henry Durrett
Date: 06:05:21 08/15/98
Go up one level in this thread
On August 14, 1998 at 11:36:56, Dan Homan wrote: <snip> >>>>>>Someone who has more statistics than I may be willing to comment on this. OK <snip> >Of course, but my point is that if you think you are learning anything >useful from 4-0 result in a 4 game match between top programs you are >wrong. I initially thought this was a great idea until I thought about >it - the more I think about it the worse it gets. <snip> (1)Every time you run an experiment you get some information. This is true even if you have only one "toss of the coin." [It just may not be what you were hoping for. You may have hoped to get nearly absolute confidence in a result you were fishing for.] (2)For example, for a given experiment [ex: so many games ("trials"), wins, losses & draws] under specified conditions [opening books used, etc.] then the results could be given as a plot of probabilities [and confidence levels] for each possible playing strength difference [expressed as rating differences.] For "Computer A is 100 points stronger than computer B" you get one probability. For "Computer A is 300 points weaker than conputer B" you get another probability. You can continue along these lines for all possible rating differences. You could plot these on a piece of graph paper. (3)Generally, the more trials, the more confidence. But this is probability, not absolute conclusions. If the weather man says 80% chance of rain, you still don't know for sure, but you will carry your umbrella anyway. (4)Small numbers of trials give very little information about extreme cases! For example, if you wish to know if computer A is 1 or 2 rating points stronger or weaker than computer B, you will have to have many trials before you can have any confidence in your conclusion. Similiarly, if you wish to know "Is Computer A is 900 points stronger than Computer B or is it 1000 points stronger" you will have to have many trials to resolve that issue with any confidence.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.