Computer Chess Club Archives


Search

Terms

Messages

Subject: Test suites vs. playing

Author: Albert Silver

Date: 17:44:31 05/24/00

Go up one level in this thread


On May 24, 2000 at 18:55:37, Marc van Hal wrote:

>On May 24, 2000 at 18:11:04, José Antônio Fabiano Mendes wrote:
>
>>http://www.brum2000.swinternet.co.uk/software.html
>He doesn't have that from a stranger I also made posts about this
>There ios a diference in solving the puzle and comming in the position of the
>puzzle

I don't agree with it and think the author misinterpreted what Bob Hyatt said.
Her is how I read Hyatt's quote "All a test suite score shows you is how well a
program does on that test suite. Doesn’t say a thing about how well it plays
games. Never has, never will."


He argues "The short reply to this is to say that an engine may not only play
games, but also analyse positions; and for many people the latter usage is more
important. The long reply would involve quibbling over the assertion that a test
suite “Doesn’t say a thing” about an engine’s game-playing strength."

The point is that a test position is one position, one ply out of a game.
A 30-40 move game has 60-80 positions. Suppose program A is capable of finding a
few moves at spectacular speeds, but it fails miserable in most others, and
program B finds those same moves a little less quickly, but finds far better
moves on the whole? Whose analysis are you going to trust? Are you going to
gamble that the positions you have program A analyze are the ones where it does
well (else it might actually suggest worse moves), or program B which can do
worse on some moves, but does better on most? Finding a mate in 12 at move 40 in
a game position in record time is nice, but what about the 39 other moves that
led to that mate?

                                     Albert Silver




The problem is that when people analyze their games, they don't look at fixed





This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.