Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Testing who is better, rating lists, and Chessmaster

Author: Slater Wold

Date: 09:41:15 07/09/03

Go up one level in this thread


On July 09, 2003 at 04:04:54, Russell Reagan wrote:

>On July 09, 2003 at 03:30:03, Dann Corbit wrote:
>
>>Once the error bars say that one cluster of settings is better, then we can
>>believe it.
>
>If the error margins indicated that one Crafty was stronger than another, would
>that mean that head to head testing is unreliable? Or does it mean anything?

You ran an interesting test, that's for sure.  But a lot of things 'come to me'
while reading your post.  The biggest:

How many games were decided stricly by the books used, and not the engine?

Think about it for a second.  If you played X amount of Crafty binaries which
are all the same exact in size, shape & form, than there is *NO* way that one
Crafty 'saw' a winning line that the other one didn't.  (Taking for granted that
this was run on a non-SMP machine.)

My point being; in the game of chess there are 3 outcomes.  Win, lose, draw.  If
you have the ability to 'guess' 99.99% of your opponents next moves, then
*theortically* you should have 99.99% draws.  If you have more, then I would
*HAVE* to guess that the book played a larger part.

Granted, computers 'stumble' into winning (or losing) positions all the time.
But 133 ELO worth?  No way.

Run the same test, first with no book at all.  Then run it using the Nunn
openings.  And see how different the three results are.  At the very least,
we'll have more to talk about.  (Or think about, anyway!)



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.