Author: Miguel A. Ballicora
Date: 08:49:16 05/11/01
Go up one level in this thread
On May 11, 2001 at 06:17:13, Dana Turnmire wrote: >BOOK OPENINGS TEST (Larry Kaufman) > >There are many ways to evaluate chess programs. They can be played against each >other, played against human opponents, or tested on problem sets. In this >article I will introduce yet another way to evaluate chess programs. The basic >idea is to compare the program's choice of move to the choices made by masters >over the years. > Here is my procedure: I first selected eight of the most popular openings >from master play. Then, using a large database of master games, I followed each >opening down the path seen most frequently at each move in the database. The >line was terminated when the sample of remaining games dropped below one >hundred. Next, each move in each line which was played at least 75% of the time >but was not an obviously forced recapture was presumed to be the best move at >that point and was marked as a problem. This produced a set of exactly one >hundred problems from opening play in which there exists a best move in the >opinion of at least 75% of masters reaching those exact positions in their own >games. > The method of testing a program is simple. First, turn off its opening book >and put it on analyze mode, or else on infinite level and monitor mode, which is >pretty much the same thing. Then, play through the opening line until a problem >position is reached, let the program think two minutes and record whether or not >the correct move is played, enter the correct move and continue to enter moves >until the next problem in the line is encountered, and repeat the procedure. >When each opening line is completed, reset for new game and go to the next >opening line. The score is simply the number of correct moves out of the one >hundred total. > The eight openings I selected were the Ruy Lopez, the Sicilian with 2 Nf3 d6, >the Winawer French, the Caro-Kann with 3 Nc3, the Slav defense to the Queen's >Gambit, the Nimzoindian defense, the King's Indian defense, and the Gruenfeld >defense. > Of course it must be remembered that this test deals only with how well a >program plays the opening without benefit of an opening book. It may be argued >that since programs do in fact play with opening books this ability is not >crucial in actual practice. On the other hand, the ability to find the best >opening moves without a book is an excelent indication of the overall strategic >and tactical abilities of the program. > The lines tested were as follows (moves in parenthesis are positions to be >tested by the program). > > In the Ruy Lopez: 1 e4 e5 2Nf3 Nc6 3 Bb5 (a6) 4 (Ba4) (Nf6) 5 (0-0) Be7 6 >(Re1) (b5) 7 Bb3 0-0 8 (c3) (d6) 9 (h3) Na5 10 (Bc2) (c5) 11 (d4) Qc7 12 (Nbd2) >cd 13 cd Nc6 14 Nb3 (a5) 15 (Be3) (a4) 16 (Nbd2). > > In the Sicilian: 1 e4 c5 2 Nf3 d6 3 (d4) (cd) 4 (Nd4) (Nf6) 5 (Nc3) a6 6 Bg5 >(e6) 7 (f4) Be7 8 (Qf3) (Qc7) 9 (0-0-0) (Nbd7) 10 g4 (b5) 11 (Bf6) (Nf6) 12 (g5) >Nd7 13 f5 Nc5 14 (f6). > > In the Caro Kann: 1 e4 c6 2 d4 d5 3 Nc3 (de4) 4 Ne4 Bf5 5 (Ng3) (Bg6) 6 h4 >(h6) 7 (Nf3) (Nd7) 8 (h5) Bh7 9 (Bd3) (Bxd3) 10 (Qxd3) Qc7 11 (Bd2) e6 12 >(0-0-0) Ngf6 13 Ne4 0-0-0 14 (g3) Ne4. > > In the French: 1 e4 e6 2 d4 d5 3 Nc3 Bb4 4 e5 c5 5 (a3) (Bc3) 6 bc3 (Ne7) 7 >Qg4 Qc7 8 (Qg7) Rg8 9 Qh7 cd 10 (Ne2) (Nbc6) 11 (f4) (Bd7) 12 (Qd3) dc 13 Nc3 >(a6). > > In the Slav: 1 d4 d5 2 c4 c6 3 Nf3 (Nf6) 4 Nc3 dc4 5 (a4) (Bf5) 6 e3 (e6) 7 >Bc4 (Bb4) 8 (0-0) Nbd7 9 Qe2 0-0 10 e4 (Bg6) 11 (Bd3). > > In the King's Indian: 1 d4 Nf6 2 c4 g6 3 Nc3 Bg7 4 (e4) (d6) 5 Nf3 (0-0) 6 >(Be2) (e5) 7 0-0 Nc6 8 (d5) (Ne7) 9 Ne1 (Nd7) 10 Nd3 (f5) 11 (Bd2) Nf6 12 (f3) >f4 13 c5 (g5) 14 Rc1. > > In the Nimzo-Indian: 1 d4 Nf6 2 c4 e6 3 Nc3 Bb4 4 Qc2 0-0 5 a3 (Bc3) 6 (Qc3) >(b6) 7 (Bg5) (Bb7) 8 f3 h6 9 (Bh4) (d5) 10 (e3) (Nbd7). > > In the Grunfeld: 1 d4 Nf6 2 c4 g6 3 Nc3 d5 4 cd Nd5 5 (e4) (Nc3) 6 bc3 (Bg7) >7 Bc4 c5 8 (Ne2) Nc6 9 (Be3) (0-0) 10 (0-0) Bg4 11 (f3) (Na5) 12 Bd3 (cd) 13 >(cd) (Be6) 14 Rc1 (Ba2) 15 (Qa4) (Be6). This is interesting to test just for fun, but it means little or nothing. There are several moves that could be replaced by others that could be equally good or they just transpose. Who is going to explain to Kramnik that Nf6 (Ruy Lopez Berlin) is inferior than a6? 6.... Ne7 in the french is ridiculous. Qc7 is almost the same and transposes after Qg4. In the Caro-Kann, Nf6 can replace Bxd3 perfectly since transposes to main variation (this trick has been used by Larsen since he wanted to deviate at a proper time). In the slav, e6 will go to the Meran systems and the like rather than Nf6 What's wrong with that? Grunfeld, nothing wrong with 8.Be3, it transposes. etc. etc. etc. The idea of the test is interesting, but it should be weed out of some of the moves that are supposed to be "the move of choice". Regards, Miguel
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.