Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Comments of latest SSDF list - Nine basic questions

Author: Rolf Tueschen

Date: 05:54:50 06/03/02

Go up one level in this thread


On June 03, 2002 at 01:42:07, Uri Blass wrote:

>On June 02, 2002 at 22:06:58, Vincent Diepeveen wrote:
>
>>On June 01, 2002 at 13:14:58, Andrew Dados wrote:
>>
>>Hello,
>>
>>of course there are problems with the rating truths.
>>
>>For example here it says someone 1000 points lower
>>rated has 0.003 chance against me.
>>
>>Or 3 out of 1000 games.
>>
>>However OTB, it is not 3 out of 1000 games. Not even 1 out of 1000.
>>It is 0 out of 1000 and nothing else.
>
>I do not know about a lot of cases when the difference was 1000 points so I
>doubt if we have enough statistic but I agree that the expected result is
>probably less than 1 out of 1000.

Uri, why are you so fixed on this? As Vincent told you, and he comes from both
directions, human chess and computer chess, the assumption of a normal
distribution in "real" strength as far as computer chess is concerned - is
simply false. It is a fata morgana. Pure nonsense. And this is the main reason
why all the pseudo-stats of SSDF is the same. Your expectancy is for human
beings but not for machines. The reason is the determinism in machines!

(Just to make a few conclusins for you, but we don't need them, because with the
absence of normal distribution it's already all over:

The main error margins they have in SSDF result from chess coincidences but not
from differences in strength. The strength is predefined and only the learning
could be better or worsen the performance. The actual versions on better
hardware will always be "better", but this isn't a test result, it was known
before. That is why we needed thousands of games between programs not just 40 to
get rid of the big margin. Then you are at Vincent's certainties. It's either
50%, 100% or 0% each for the three possible results in chess. The expectancy
itself is always 1 for one of the three priorily known and defined results,
either equal or weaker or better  - period. So, over thousands and millions of
games, the few new versions of a season will be equal, what is already clear for
someone who knows about stats when he's looking at the SSDF ranking list. The
new progs are always better than the older ones, no matter if with a result of
75% or 87%. But these differences do _not_ define differences in strength
between the new programs itself. Their results against each other would always
be equal after big, huge matches. Of course we have several categories. The best
and the second category etc. But also this is priorily well defined. The rest is
all bogus.)

Copyright for the final paragraph above by Rolf Tueschen June 2002

Rolf Tueschen



>
>>
>>Idem for up to 600 points.
>
>I know about at least one case when 1900 boy won a game against 2500 GM.
>I believe that the expected result is less than 3% but I have no statistic about
>games when the difference is more than 600 elo.
>
>Note that the real difference in that games is usually smaller because strong
>1900 players get more often the opportunity to play against GM's when the swiss
>system is used.
>
>>
>>The rating idea doesn't work when a player who is titled
>>plays a < 1700 player simply.
>
>Another point is that statistic here may be misleading because the <1700 player
>who wins or draws against a master may be really 2000 player who trained against
>computers and not <1700 player.
>
>Uri



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.