Author: Dana Turnmire
Date: 03:17:13 05/11/01
BOOK OPENINGS TEST (Larry Kaufman) There are many ways to evaluate chess programs. They can be played against each other, played against human opponents, or tested on problem sets. In this article I will introduce yet another way to evaluate chess programs. The basic idea is to compare the program's choice of move to the choices made by masters over the years. Here is my procedure: I first selected eight of the most popular openings from master play. Then, using a large database of master games, I followed each opening down the path seen most frequently at each move in the database. The line was terminated when the sample of remaining games dropped below one hundred. Next, each move in each line which was played at least 75% of the time but was not an obviously forced recapture was presumed to be the best move at that point and was marked as a problem. This produced a set of exactly one hundred problems from opening play in which there exists a best move in the opinion of at least 75% of masters reaching those exact positions in their own games. The method of testing a program is simple. First, turn off its opening book and put it on analyze mode, or else on infinite level and monitor mode, which is pretty much the same thing. Then, play through the opening line until a problem position is reached, let the program think two minutes and record whether or not the correct move is played, enter the correct move and continue to enter moves until the next problem in the line is encountered, and repeat the procedure. When each opening line is completed, reset for new game and go to the next opening line. The score is simply the number of correct moves out of the one hundred total. The eight openings I selected were the Ruy Lopez, the Sicilian with 2 Nf3 d6, the Winawer French, the Caro-Kann with 3 Nc3, the Slav defense to the Queen's Gambit, the Nimzoindian defense, the King's Indian defense, and the Gruenfeld defense. Of course it must be remembered that this test deals only with how well a program plays the opening without benefit of an opening book. It may be argued that since programs do in fact play with opening books this ability is not crucial in actual practice. On the other hand, the ability to find the best opening moves without a book is an excelent indication of the overall strategic and tactical abilities of the program. The lines tested were as follows (moves in parenthesis are positions to be tested by the program). In the Ruy Lopez: 1 e4 e5 2Nf3 Nc6 3 Bb5 (a6) 4 (Ba4) (Nf6) 5 (0-0) Be7 6 (Re1) (b5) 7 Bb3 0-0 8 (c3) (d6) 9 (h3) Na5 10 (Bc2) (c5) 11 (d4) Qc7 12 (Nbd2) cd 13 cd Nc6 14 Nb3 (a5) 15 (Be3) (a4) 16 (Nbd2). In the Sicilian: 1 e4 c5 2 Nf3 d6 3 (d4) (cd) 4 (Nd4) (Nf6) 5 (Nc3) a6 6 Bg5 (e6) 7 (f4) Be7 8 (Qf3) (Qc7) 9 (0-0-0) (Nbd7) 10 g4 (b5) 11 (Bf6) (Nf6) 12 (g5) Nd7 13 f5 Nc5 14 (f6). In the Caro Kann: 1 e4 c6 2 d4 d5 3 Nc3 (de4) 4 Ne4 Bf5 5 (Ng3) (Bg6) 6 h4 (h6) 7 (Nf3) (Nd7) 8 (h5) Bh7 9 (Bd3) (Bxd3) 10 (Qxd3) Qc7 11 (Bd2) e6 12 (0-0-0) Ngf6 13 Ne4 0-0-0 14 (g3) Ne4. In the French: 1 e4 e6 2 d4 d5 3 Nc3 Bb4 4 e5 c5 5 (a3) (Bc3) 6 bc3 (Ne7) 7 Qg4 Qc7 8 (Qg7) Rg8 9 Qh7 cd 10 (Ne2) (Nbc6) 11 (f4) (Bd7) 12 (Qd3) dc 13 Nc3 (a6). In the Slav: 1 d4 d5 2 c4 c6 3 Nf3 (Nf6) 4 Nc3 dc4 5 (a4) (Bf5) 6 e3 (e6) 7 Bc4 (Bb4) 8 (0-0) Nbd7 9 Qe2 0-0 10 e4 (Bg6) 11 (Bd3). In the King's Indian: 1 d4 Nf6 2 c4 g6 3 Nc3 Bg7 4 (e4) (d6) 5 Nf3 (0-0) 6 (Be2) (e5) 7 0-0 Nc6 8 (d5) (Ne7) 9 Ne1 (Nd7) 10 Nd3 (f5) 11 (Bd2) Nf6 12 (f3) f4 13 c5 (g5) 14 Rc1. In the Nimzo-Indian: 1 d4 Nf6 2 c4 e6 3 Nc3 Bb4 4 Qc2 0-0 5 a3 (Bc3) 6 (Qc3) (b6) 7 (Bg5) (Bb7) 8 f3 h6 9 (Bh4) (d5) 10 (e3) (Nbd7). In the Grunfeld: 1 d4 Nf6 2 c4 g6 3 Nc3 d5 4 cd Nd5 5 (e4) (Nc3) 6 bc3 (Bg7) 7 Bc4 c5 8 (Ne2) Nc6 9 (Be3) (0-0) 10 (0-0) Bg4 11 (f3) (Na5) 12 Bd3 (cd) 13 (cd) (Be6) 14 Rc1 (Ba2) 15 (Qa4) (Be6).
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.