Author: Kirill Kryukov
Date: 05:02:37 01/05/06
Go up one level in this thread
Hi George, Thanks for comments! I know there are two big camps of people here - those who test with own book and those prefering generic books. We test with generic book because we are interested in the playing style and strength of the engine itself. Large opening book hides the engine performance until the middlegame, and sometimes well into the endgame, while it's the opening stage where the most critical decisions are made. By using generic books, and also relatively short ones, we are performing more clean and scientific comparison of the pure engines, which is also more profound. On January 05, 2006 at 07:21:54, George Tsavdaris wrote: > That is shame in my opinion. Of course is a matter of taste and personally, i >always prefer own engine books to be used! I don't understand why most lists >doesn't use own engine books and instead use some general books or position >sets?!?!? That's why i like SSDF(not only for that of course). I'll tell you some reasons why we and many people test with generic book: 1. Opening book quality is directly proportional to the amount of money invested into the chess program. Engine strength, on the other hand, is proportional mostly to the talent and efforts of the programmer. 2. Opening book can easily be a team work, most engines are creation of a single guy. 3. Opening book is about memorization, engine is about playing chess. 4. Many amateur engines have no any reasonable book, since their authors can't afford to hire a chess professional to create and tune one. Such engines will be in big disadvantage, even if the author is genius and invested lot of work. 5. If we publish list of engines with own books, it will help to the marketing of commertial chess programs (Particularly those from Chessbase). I would much rather help to the amateur authors. Because of all those points, I actually think it's a shame to test with own book, but it's OK to have such studies too, because we can learn more from comparison. > Of course when using general books, the list is again interesting to watch, so >that new list will be interesting to follow too.....! Yay to that! > Also by using a really good, for the following purpose generic book, is good to >measure the pure engine's strength in general, that is in all kind of >situations. But this perhaps, may lead to wrong conclusions. > For example let's say your list with a general book shows Engine-A is better >with +60 ELO from Engine-B. Now someone may believe that Engine-A is a lot >stronger than Engine-B in general. But perhaps if Engine-B plays with its own >book, then there is the possibility Engine-A(that plays with the generic book) >to be even worse than engine-B. Also there is the possibility Engine-A even if >it played with its own book to be again worse than Engine-B that plays with its >own book. > I've put a "perhaps" in all these, as no one so far has proven such a thing, >but also no one has proven the opposite..... We realize that results of engines A and B tested with generic book can be very different from their results with own books. We don't speculate about engine performance with own book based on tests with generic book, although of course there is good correlation. Also please note that for any ordinary chess player, who is using chess engine to analyze positions, prepare opening schemes, etc, the engine's performance with generic book is much more important. >Any link? Is the http://kd.lab.nig.ac.jp/chess/engine-tourney/ >the correct one.....? No link yet, sorry. That link is not the correct one. Best, Kirill
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.