Author: David Dory
Date: 02:54:34 02/17/03
Go up one level in this thread
On February 16, 2003 at 10:38:01, Albert Silver wrote:
>>This is the way Rolf believes the SSDF list should be ordered:
>>(From a previous post in this thread)
>>"
>>it would help a lot if you would take a look into some stats readers. I am not
>>inventing opinions, I am talking about simple basics, laymen often forget. That
>>is also why it is not possible to adopt a certain routine out of the context of
>>a whole method and then do some calculations and whoopie preseting the winner. I
>>repeat the actual results - given in the list itself - does NOT allow to
>>present Shredder as number one. Also: if SSDF understood what they were doing
>>they had done a presentation of the sort 1.-3. are... Then Shredder could well
>>be named in the first place. 1.-3. Shredder, Fritz, Fritz
>>
>>Rolf Tueschen
>>"
>>Of course, when the number and result of games could statistically separate the
>>programs strength reasonably beyond the margin of error, then a #1, etc., could
>>be and should be dutifully reported.
>>
>>Dave
>
>I had hoped he had something more interesting in mind. All the above says is
>that the SSDF shouldn't publish their results until 10,000 games have been
>played OR present the results as Tony Hedlund jokingly replied.
>
>I think it's nonsense. The SSDF is simply an organization that has programs play
>hundreds of times against each other and they present the latest results
>periodically. They provide all data so that people know how reliable these
>comp-comp ratings are: match results, number of games, and the error margin. For
>some reason people constantly go gung-ho if they don't like the results. Rolf
>claimed they aren't scientific as they aren't absolute, and can change in time.
>As far as I know, the SSDF *never* claimed these results were immutable and set
>in stone. They are merely the latest results. No more, nor less. As always.
>
>Take the FIDE ratings for example. Do they never change? Should we attack them
>for being non-absolute? There is even the alternate rating list with different
>rankings for the top players. What about that? Attack it to? If the data wasn't
>deliberately modified and the parameters and methodology are transparent I don't
>see what the problem is.
>
> Albert
Have you ever known Rolf NOT to have a problem with the way something is? :)
Seriously, his stated problem is the (over) emphasis on marketing, ("we're
number 1 on SSDF!), which he feels is used to persuade people to buy new chess
programs, unnecessarily.
I see what he's talking about - it does somewhat distort the perception of
strength in a new chess program, when in fact, said program may NOT be the
strongest program after further testing. I think the average consumer is not
going to take the margin of error into account, or understand how the ratings
may change for a new program on SSDF.
I'm sure it's no problem for folks on CCC, especially.
Dave
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.