Author: Stephen A. Boak
Date: 16:24:25 11/09/02
Go up one level in this thread
On November 09, 2002 at 12:02:26, Uri Blass wrote: >This test suite is a joke because programs can often solve >it for the wrong reason. > >The problem is not the fact that the >test is to avoid moves but the fact >that programs can often avoid the move for the wrong reason. > >Uri hi Uri, The probability that a somewhat weaker engine will avoid the move for the wrong reason in an abnormally large number of avoid move problems is not likely statistically. I'm not talking about the worst kind of engines--the ones that evaluate or calculate so poorly that the move to be avoided is never looked at; or, if it is looked at it, never looks attractive (in evaluation) to the engine. Obviously such poor engines appear to be more like random movers, and the odds of avoiding a normally attractive 'avoid move' would be high. Quantity forty (40) avoid move problems gives a good relative test among programs. The relative test is not of overall program strength, Uri, but as others indicated, only a test of how the program does on *that* type of problem test. If a known very weak engine got a perfect 'avoid move' test score, I would never extrapolate (wrongly) from that result that the engine was the best at playing a full game of chess. That would be ludicrous. The proper use of the test is to discriminate among programs to see their level of ability to 'avoid' bad moves. Nothing more, nothing less. The probability that the winner of the 'avoid move' test is also the best chess program also depends on many other important factors--such as the ability to pick the 'best move' relatively more often than its competitors. Don't read into the 'avoid move' test more than it provides. More importantly, don't assume that most readers will read into the test more than it provides. Regards, --Steve
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.