Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: SSDF Rating List 2006-01-03 - no longer acceptable !

Author: Dann Corbit

Date: 09:21:40 01/04/06

Go up one level in this thread


On January 04, 2006 at 11:13:32, Eduard Nemeth wrote:

>>>Next: Thoralf says that Fruit will played too agains weaker engines, but in my
>>>opinion should test next Programs like Hiarcs 10, Rybka 1.2, SmarThink 1.0 and
>>>Ktulu 7.5.
>>>
>>>The Freature is also: Not against weaker Engines but against stronger Engines.
>>>
>>>That is for me the reaso why I see this Rating List as a Joke - real!
>>>
>>>ED.
>>
>>I do not agree.
>>
>>part of the ability of chess program is in beating weak programs and if Fruit
>>beats Nimzo8(450 mhz) more convincingly than Fritz then it is something positive
>>about fruit.
>>
>>Uri
>
>Uri, see the all matches and You will see: Fritz 9 vs. Fritz532 (on P200!) 3-1.
>For me is this a Joke. Why play Fritz vs Fritz? Why not against the ChessMaster
>before? Many Things (in my eyes) are mysterious by SSDF.

There is simply no doubt that the SSDF is the very best data available to tell
us the strength of programs.

There is nothing I can see that is definitely wrong with their methodology.
Often, the complaints against them are very misguided.  We should *expect* some
strange match results.  But with hundreds of games those things even out.  And
the use of opponents that are 200 Elo weaker or stronger is not a serious defect
(but huge Elo differences would be bad.)

Of course, as with any experimental setup, it could be possible to improve it.
But there is nothing better to tell us a more accurate answer.  For certain, the
SSDF tells us far more than contests like WMCCC and Paderborn.



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.