Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: 9 rounds will not always give you the "best" program

Author: Sune Fischer

Date: 15:41:02 01/20/03

Go up one level in this thread


On January 20, 2003 at 18:34:21, Dann Corbit wrote:

>On January 20, 2003 at 18:08:45, Sune Fischer wrote:
>
>>On January 20, 2003 at 17:27:44, Dann Corbit wrote:
>>
>>>>>No contest can truly tell us which program is strongest.  Not even a trillion
>>>>>rounds of round-robin.
>>>>I disagree again.  I believe a trillion rounds will show which program is
>>>>strongest.
>>>
>>>You're wrong.
>>>
>>
>>No he is right.
>>There is a saying in statistics (IIRC correctly) "null events don't happen".
>>
>>Basicly it means things that are very very improbable are impossible.
>>
>>You would never see TSCP beat Fritz more than 50% of the time if you did a
>>trillion games. No one has done more than a trillion games yet, we all know
>>fritz is stronger, why is that? ;)
>
>Until the number of games reaches infinity, there will always be uncertainty.
>
>Because there is some degree of randomness in the programs, I'm not even sure
>that there *is* an answer to the question:
>"Which is stronger, Chess Tiger or Fritz?"
>
>For programs with hundreds of ELO difference, you can be fairly certain
>relatively quickly.  For programs of about the same strength, you will never
>know the answer.

But what you were saying was, that you could _never ever_ know the answer. There
is a fundamental difference I think and this is where the null event theorem
saves us. It _is_ possible to make an accurate statement if you have reduced it
to a null event. After 1 trillion games I think we have a clear winner, whom
ever that may be.

-S.



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.