Author: Dana Turnmire
Date: 06:15:42 05/29/01
This article appeared in the 1996 issue of Computer Chess Reports by IM Larry Kaufman. "In past years I have occasionally written about the problem of Computer opening books being designed especially to defeat specific competing programs, so as to outrate them on the various Computer rating lists, especially the "Ply" list from Sweden. I know this was done by many if not all of the dedicated manufacturers in the past; in fact I even did a bit of it myself for Novag several years ago on its "Super-Expert" models. Since there were so many different models, the effect of such rigging was always rather small on the ratings, since it was not practical to rig against more than a few opposing machines. Now that the top PC programs can defeat nearly all opposition except each other, the problem has resurfaced since only two or three main rivals need to be booked against. According to an article in the new issue of "Selective Search", MChess Pro 5.0 has set new records in terms of rigging its book to defeat its opposition, in particular the "Genius" programs. In one game Genius left book on move 17, while MCP5 remained in book until move 39(!) by which time it was clearly winning. Even with the book off, MCP5 makes its moves instantly until move 39 in this line, which means that the program itself contains a "hidden" book separate from the normal book. In another game, Genius left its book on move 10, while MCP5 remained in book until move 24, at which point it thought and then announced mate in 7! As a result of this "booking", MCP5 achieved crushing scores against its leading competitors and got the top spot in the "Ply" rating list. My point here is not to disparage MCP5, which is a fine program. The book author undoubtedly felt justified in putting in such killer lines on the grounds that other programmers also do this; he merely did it better and more pervasively than others. One effect of this type of rigging is that newer programs always get higher ratings than the older ones, by more than they deserve, since the new programs can only be rigged against older ones. Years ago I argued in favor of "reversal" testing in which the opening lines are selected form a list of standard ones, with each computer playing the line once as white and once as black. While there are arguments against this (is it fair to force a program to play an opening contary to its style?), it seems that now the merits of this reversal testing outweigh the demerits, and I recommend that future testers consider switching to thie methodology."
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.