# Computer Chess Club Archives

## Messages

### Subject: Re: A question about statistics...

Author: Ricardo Gibert

Date: 10:56:03 01/04/04

Go up one level in this thread

```On January 04, 2004 at 12:52:46, Mark Young wrote:

>On January 04, 2004 at 12:40:00, Ricardo Gibert wrote:
>
>>On January 04, 2004 at 12:29:15, Mark Young wrote:
>>
>>>On January 04, 2004 at 11:46:00, Roger Brown wrote:
>>>
>>>>Hello all,
>>>>
>>>>I have read numerous posts about the validity - or lack thereof actually - of
>>>>short matches between and among chess engines.  The arguments of those who say
>>>>that such matches are meaningless (Kurt Utzinger, Christopher Theron, Robert
>>>>Hyatt et al)typically indicate that well over 200 games are requires to make any
>>>>sort of statisticdal statement that engine X is better than engine Y.
>>>>
>>>>I concede this point.
>>>
>>>If you concede this point you don't understand. There is no magic number like
>>>200 or 2000. The score must be considered. Here is an example:
>>>
>>>A score of 17 - 3 in a 20 game match has a certainty of over 99% that the winner
>>>of the match is stronger then the loser.
>>>
>>>A 100 game match ending 55 - 45 only has a 81% chance that the winner of the
>>>match is the stronger program.
>>>
>>>A 200 game match ending 106 - 94 only has a 78 % chance that the winner is
>>>stronger then the loser.
>>
>>
>>Nothing you have said is really correct because you have ignored the significant
>>effect of draws in a match.
>
>I can only say WHAT!! The last time I checked wins count as 1 point, draws count
>as 1/2 point, and loses count as 0.
>
>So I have no clue what is going on in your brain to make such a comment!!
>
>In a 20 game match winning with 17 wins and 3 losses  0 draws is equal to
>winning with 14 wins 0 losses and 6 draws. You win both matches 17 - 3. The
>results are one in the same.
>

Have look at my response to Peter Berger:
http://www.talkchess.com/forums/1/message.html?340175

```