Author: Peter Fendrich
Date: 08:27:21 06/11/01
Go up one level in this thread
On June 11, 2001 at 11:07:31, Bas Hamstra wrote: >I think this is how it works: if both programs are equally strong, they you can >expect they both would score 50% in a match. However we have noise and luck. >Therefore in a 100 game match (still supposing both are equally strong) they >outcome will not be exactly 50%, but will lay within a "window". > >Now you can say something about that window. For instance that it will stretch >from [50% - 3Sigma, 50% + 3 * Sigma] 99.8% of the time. Same goes for 2*Sigma, >accurate 95% of the time. > >Sigma is to calculate here as SQR(n * p * q) where n is the sample size, p is >A's winning chance (0.50) and q is B's winning chance (also 0.50). > >So lets take a 100 game match. Sigma would be SQR(100 * 0.50 * 0.50) = 5. >Therefore the noise window is [45, 55] with 95% confidence. If the outcome falls >OUTSIDE this window, it is fair to say the proggies are not equally strong, in >other words, one is better. > > > >Bas Hamstra. What you are using here is the Normal distribution. Game results does not have that distribution. The real distribution is the Trinomial. The Normal distribution can only be used as an apprapproximate value when enough number of games are played (maybe 30 or so is the lower limit). With fewer games played it will not only give wider confidence intervals, it is even not accurate to use as an approximation of the Trinomial distribution. Your 100-game example is OK of course but the table covered also as few as 10 games. //Peter
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.