Author: Mike S.
Date: 13:50:53 10/22/03
Go up one level in this thread
On October 22, 2003 at 14:21:34, Mogens Larsen wrote:
>On October 22, 2003 at 12:50:01, Mike S. wrote:
>
>>Actually it would be *very unfair* to test King with default settings against
>>other top programs, because it would limit it way below it's potential, on fast
>>comps (estim. 500 MHz...3 GHz) typically used nowadays among computerchess fans.
>
>Irrelevant.
It is *irrelevant* to you, if testing is fair or not??
Again: Testing The King with CM defaults would mean, to intendedly (!) limit it
to a level much lower than it's potential, which has been shown by these custom
settings. You've seen my example from the Grailmaster ratings. The difference is
more than 100 Elo.
Generally, I have the same "out of the box" testing philosophy. But every rule
has it's exception. Do you really mean, it makes perfect sense to test
Chessmaster defaults, when we *know* that custom settings are up to 100 elos
better in computer chess, or even more? Who should be interested in such test
results?
>(...) In short, the tested product should be recognizable to the average user without
>special knowledge. Especially if purchase is motivated by the testresults. It's
>product testing, nothing else.
SSDF doesn't test for Auntie Clara who buys Chessmaster for her 8-year old
nephew's birthday. These people don't read SSDF and need neither SSDF nor
plus/minus 100 elos. The CM defaults have a type of average (US) users in mind,
who probably have slower old computers than the average computerchess fan,
especially the european fan (maybe Sel. 9 is better on much slower computers, I
would even doubt it).
But within the SSDF audience, only ignorants use CM defaults. Should SSDF test
for ignorants or for informed computerchess experts? Also, it would shade a
"doubtful light" so to speak, on the SSDF top ranking engines - which it does
currently too when King 3.23 is missing - because we don't know how their
results would change when the have to play King 3.23 running with proper
settings. Some will score better, some worse. This may change rankings
significantly, since differences are small.
But actually, it's not such a big concern of me than it may seem :-)), because
there are several other rating lists which include King, among all the other top
engines:
The King 3.23 currently ranking...
CSS ratings by Klaus Wlotzka #6
CSS blitz ratings by Stefan Pohl #4
FRC rating list by Stefan Pohl #4
rating lists by Gerhard Sonnabend #5 to #9 (incl. Deeps)
All these use Sel.=12
Regards,
Mike Scheidl
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.