Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Junior - Crafty NPS Challenge - a user experiment

Author: Rémi Coulom

Date: 09:59:39 11/24/03

Go up one level in this thread


On November 23, 2003 at 19:26:35, Sune Fischer wrote:

>On November 23, 2003 at 14:31:43, Dieter Buerssner wrote:
>
>>On November 22, 2003 at 20:01:27, Robert Hyatt wrote:
>>
>>>I disagree.
>>
>>Ditto
>>
>>>6-0-0 vs 6-0-1000 are way different results.  And the
>>>rating and rating error bar would be far different.
>>
>>I tried to explain, that Elo rating is not an objective measure for the
>>likelyhood, that one is better.
>
>Elo doesn't try to measure the likelyhood at all, that is the problem.
>
>Anyone who understands real world numbers knows that they don't make a lot of
>sense without some knowledge of their tolerance.
>
>>>With a 6 0 result
>>>I would conclude the 6 side is significantly better.  with 6 wins and
>>>1000 draws I would not conclude _either_ was better with any confidence.
>>
>>Both resutlts are identical for the question for the likelyhood, who is better.
>
>Without having read his paper I'd say that a 6-0 score indicates the winner is
>far better than the loser, but the confidence is very low.
>
>Where as a 10006-10000 result indicates the players are almost equal with a very
>high confidence.
>
>I don't think that doesn't necessarily contradicts what you say though.

No, it does not.

>
>>If I cannot convince you, perhaps have a look at Rémi Coulom's paper, available
>>from http://remi.coulom.free.fr/ (inside
>>http://remi.coulom.free.fr/WhoIsBest.zip). One cite from that paper:
>>
>>"This proves that the likelihood that the first player is best does not depend
>>on the number of draws."
>
>Something to read tonight perhaps :)

The paper is a bit mathematical, but the fact that the likelihood does not
depend on the number of draws can be explained intuitively rather easily:
imagine a game called "chess+" where no draw is possible: each time a game is
drawn, the two players start over from the initial position until one player
wins. Draws are not counted. For the exact same sequence of games, depending on
whether you consider they play chess or chess+, the score will be 1006-1000 or
6-0. Obviously, the likelihood that one is better than the other is the same.

Of course, this is true only if the hypotheses are true: games are independent
random events, and the prior is uniform (which is reasonable in comp-comp
matches without learning).

I hope this message will save some mathematical reading for some.

Rémi

>
>-S.
>
>>Regards,
>>Dieter



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.