Author: Dann Corbit
Date: 13:10:56 01/29/99
Go up one level in this thread
On January 29, 1999 at 15:48:15, KarinsDad wrote:
[snip]
>The topic is: How can we find out if programs (not humans) have been getting
>better? Of course, someone could write a better opening book and a better
>tablebase. No question about it. And yes, this does happen with the commercial
>programs.
>
>But if you want to compare engine strength versus engine strength to determine
>if the program got better and not compare whether some database has better data
>in it, then you should use the same databases.
I disagree. What you are testing here is engine strength verses engine
strength. That is one particular measure of the ability of a chess engine. But
that same engine will play *better* with an excellent opening book. And *better
again* with an excellent endgame tablebase. So that is only a subset of a
program's ability to play the game of chess.
>If I could take a 1200 rated playing program and give it an opening database of
>all moves out to 100 moves for each side (a very large database on the magnitude
>of 10^320 positions) and a tablebase which can handle all positions with 12
>pieces on a side (another extremely large database which I cannot even guess how
>to calculate), then I would have a program that would never lose to Deep Blue
>since it would never use it's search engine for anything other than looking up
>data out of databases.
Well, if you could produce such a remarkable database system, then your program
would have that ability. It does not matter where the answers come from.
Should we get annoyed because the computer did not have to think about it but
instead did a simple lookup? A chess program is a black box. Into it go board
positions and out of it come board positions. How it generated the positions is
not relevant in determining the strength of the program. If we remove all
database entries, then we measure only engine strength.
>Would CM6000 be stronger than CM5000 with a stronger opening database? Most
>likely. Is it a fair test to compare CM5000 with CM6000 with them both using the
>same opening database? Of course. That's the point. If CM6000 has an inferior
>engine to CM5000, but had a much more superior opening book, it could still win
>games due to being in a superior position out of the opening.
No more fair or less fair than testing with different database or endgame
tablebase systems or whatever. You should describe in the test the full nature
of the variables of the experiment, but if you want to find out how well a
program plays chess, you do not remove the data it normally has at its disposal.
It will play far worse than it is capable of.
>The difference between humans and programs is that the opening book of a human
>is an integral part of him whereas this is not the case with a program. A
>program can use any opening book (in the appropriate format) or none at all. You
>cannot compare the two.
A human's opening book also changes over time. You can learn new openings or
you can forget how to use an opening you have not used for a while. You can
also have holes in your opening book just like a computer.
Furthermore, *my* opening book is a microscopic fraction of the opening book of
a GM. Is it fair to have us play against each other when his opening book is
much larger? Of course it is. If I want a bigger {internal} opening book, I
should study more. And if I simply lack the capability to gather an opening
book the size of Sierwan or Karpov or whomever, then tough -- I just have to get
along with what I am capable of mustering.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.