Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Dangers in CC - SSDF: Terminology, Statistics

Author: Maurizio De Leo

Date: 03:43:20 02/21/03

Go up one level in this thread


Rolf, it seems that for one time you are using topics instead of personal
attacks, so I will answer you.

I agree with your point : there must be a "measurement unit". In races that is
0.01 seconds, in ski it is 0.001 sec.

In SSDF that is **** 1 SSDF point *****

The program EloStat calculates rating which are integer number, even if real
rating based on the result should have been 2713.56 or 2689.22

I don't know if it has happened in the past, but I'm pretty sure that if 2
programs finish in the list with the exactly same score, the SSDF would put them
"ex-aequeo", as it's done when at the olympiad two athletes have the exact same
time till to the measurement unit.

So, see the SSDF like a competition : who arrives first, even for a millisecond
or for a point, wins.


A COMPLETELY different things is to decide "who is the best".

Not always who wins olympiad is the best. For example the best could have had a
little stomacache during the run, or in his lane the wind was blowing a tad more
fast against him or so on.
Of course if someone wins a lot of competitions, you can say : "I'm xx percent
sure that he is the best"

Same apply for the SSDF. Shrederr has "won the race", doesn't matter if it had a
1 or 100 points advantage. It is so the SSDF number 1.

For who is the best......it all depends on your preferred confidence range. If
you use the 95% standard the best is "one of the first 5 programs".

But you can also change the confidence range, and this will shrink or enlarge
the intervals. So you can easily say "I'm 20% or 25% sure that Shredder is the
best"

Of course if you want certainity you can also never have a leader. For 99%
certainity the intervals will become around +-50 points while for 100% all you
can say is "one of the contestant is the best" :-)


So, bottom line is that Shredder is "SSDF number 1". It won the contest. For
which is the best program, you should rely on an educated guess.


Maurizio






This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.