Author: Will Singleton
Date: 20:34:15 08/26/02
It's common wisdom that a good book will increase a program's strength, and I have no doubt about it. But, what about the typical amateur book, which is usually created from pgn files, without a great deal of hand tuning? I don't know much about chess, but I have observed that most programs don't have a clue to what's happening in the opening as it pertains to the final outcome of the game. Chess is just too complex for that. So, could a crude book, created from pgn files, significantly help the typical amateur program? I felt I should conduct a small test involving my program, Amateur. Its book is a compilation of generally available large pgn collections, and relies on won-loss percentages to select moves. A random factor is applied to choose reasonable moves. Against typical competition, it will leave book in 10-20 moves, with a good percentage of viable positions. In short, it's an OK amateur book. I first played a match between Amateur 2.1 with book vs the identical program without book. Using a short time-control of 3/1, I was surprised to see that the results of a 20 game match were about even. I attribute that to the fact that the book version was soon out of book after the non-book version played some non-book move. Then I played two matches using another program. Amateur played Beowulf with and without book. This was even more surprising to me. The non-book version of Amateur lost to Beowulf 2-7-1. The book version beat Beowulf 6-2-6. I think one needs to have a larger sample of games to really have an opinion about the issue. I am not sure that the book plays a critical role if, upon, leaving book, the programs have a more or less even eval. But perhaps that's deceiving. Perhaps the won-loss percentages are the better indicator of success than the program's eval, even after 15-20 moves. If that's the case, then how can the eval be adjusted to account for this? Or should it? I don't really know how to improve my book, or if the results I got are significant. I think my next step is to make a huge book using some of the commercial pgn collections, then test that against my present book. But I think one thing is clear: a really good book is extremely difficult to make, and if the commercial gui's have very good books, then programs that use them will have a huge advantage. Questions for other researchers: in book testing, how soon is the book program out of book? If it is early, and without a significant advantage at that point, what does that mean for the test? How does one avoid duplicate games? I avoided that by having the "non-book" version play a maximum of 3 moves from book. This usually results in different openings, especially given a small test sample. Will
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.