Computer Chess Club Archives




Subject: Re: A question about statistics...

Author: Roger Brown

Date: 09:48:52 01/04/04

Go up one level in this thread

On January 04, 2004 at 12:29:15, Mark Young wrote:

>On January 04, 2004 at 11:46:00, Roger Brown wrote:
>If you concede this point you don't understand. There is no magic number like
>200 or 2000. The score must be considered. Here is an example:
>A score of 17 - 3 in a 20 game match has a certainty of over 99% that the winner
>of the match is stronger then the loser.
>A 100 game match ending 55 - 45 only has a 81% chance that the winner of the
>match is the stronger program.
>A 200 game match ending 106 - 94 only has a 78 % chance that the winner is
>stronger then the loser.

I hear you.  However the point that the 200 games advocates make is that, with
respect to the 17-3 score over a 20 game match, the result is open to question
because another batch of twenty games could produce an entirely different

Kurt Utzinger published breakdown figures in match between Gandalf and a beta
engine he is/was testing and it ended up statistically even.  Had he stopped it
earlier, he would have erroneously concluded that X was stronger than Y.

I have observed that myself as well...


This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.