Author: Dana Turnmire
Date: 08:19:14 05/31/01
Go up one level in this thread
On May 31, 2001 at 04:46:10, Ricardo Gibert wrote: >On May 30, 2001 at 13:57:54, Dana Turnmire wrote: > >> This article appeared in the 1996 issue of Computer Reports and was written >>by Michael Byrne. It shows me the only fair way to find an engines true >>strength is WITHOUT the opening book. > >[snip] > >I disagree. If you compare programs without their book, you will create a >different kind of problem. A programs eval can aslo be tuned with respect with >the book they have. In other words, the book steers the position towards the >type of position that the programs eval can assess accurately. This is perfectly >legitimate. Human players do the same. As a human player, I try to play openings >that steer the game towards positions that suit my style. If you test without >book. The programs will end up in types of positions they would normally not get >into. You will not get a true measure of their playing strength. What you will >measure is the ability of program to analyze an arbitrary position. This is >worthwhile to know if that is how you are going to use a program, but it should >not be confused with playing strength. They are not really the same. > >If a program is susceptable to "rigged" opening books, that is the programs >weakness and is fair game to take advantage of. Such programs should be enhanced >so that they are less susceptable to such an attack. The way programs are >currently tested and assessed encourages programmers to develop countermeasures. >From this point of view, it is clear that the current way of testing is really >just fine. Creating an artificial setting to test programs will only result in >artificial results playing strength-wise. I suppose you have a point but if a commercial program is going to play a MATCH with a grandmaster it should be required to use its normal opening book and learning features without a team of openings experts manipulating the book for every game. If a software maker is going to advertise that it's program beat a grandmaster in a match it at least should be honest enough to let the public know the books were "cooked" for that match.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.