Author: Bob Durrett
Date: 08:33:13 10/22/02
Go up one level in this thread
On October 22, 2002 at 09:43:08, Chuck wrote: >On October 22, 2002 at 09:14:15, Bob Durrett wrote: > >>On October 22, 2002 at 08:31:34, Chuck wrote: >> >[snip] >> >>Summary: >> >>I propose that each chess engine be given the opening book which provides >>optimal performance for that engine. The chess engine programmers should have >>the final say on which book is to be used by their chess engine. >> >>Bob D. > >My reply to this is that I would like to agree, but there is a large difference >in the quality of some of the books. It is also true that there is a large difference in the quality of some of the chess engines. It seems to me that it is the responsibility of the engine programmer to produce a high-quality engine. Similarly, it should [IMHO] be the responsibility of that same programmer to produce [or provide] a quality opening book. If the programmer fails to do what he is supposed to do, then he should have to watch his product perform poorly. Of course, that would not help if one wished to evaluate the engines themselves for some reasons. But performance of an engine depends, to some degree, on the opening book. Maybe you should run the same tournament a dozen times, each time using a different book. For example, if you have 12 engines in your tournament, you could use engine #1's book for the first tournament, then use engine #2's book for the second tournament, etc. That would allow evaluation of the correlation between engine performance and choice of book. Maybe you could even do an "analysis of variance" evaluation. Such a multi-tournament event might take a little longer. : ) Bob D. >Also, some engine books give the appearance >the author has "booked up" against other engines. But that is exactly what happens in human tournaments. In each game, each player deliberately choses an opening which he/she hopes will give the best chances against that particular opponent. The opening struggle is part of the game of chess! Both sides are trying to get into a favorable middlegame. [Except, possibly, for all-out king attacks such as in pre-Morphy games.] I would be interested to know whether or not the programmers are allowed to change or modify the books between rounds in the SSDF tournaments. Bob D. >All this skews the ability to >distinguish which engine actually does a better job of evaluation. > >Key point: > >In order to be fair, the assumption should be made that we want to assess an >engines ability to evaluate chess as a whole, and not merely the subset of the >game which it's author feels it evaluates well. > >Regards, > >Chuck
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.