Author: Uri Blass
Date: 16:54:57 11/09/02
Go up one level in this thread
On November 09, 2002 at 19:24:25, Stephen A. Boak wrote: >On November 09, 2002 at 12:02:26, Uri Blass wrote: > >>This test suite is a joke because programs can often solve >>it for the wrong reason. >> >>The problem is not the fact that the >>test is to avoid moves but the fact >>that programs can often avoid the move for the wrong reason. >> >>Uri > >hi Uri, > >The probability that a somewhat weaker engine will avoid the move for the wrong >reason in an abnormally large number of avoid move problems is not likely >statistically. > >I'm not talking about the worst kind of engines--the ones that evaluate or >calculate so poorly that the move to be avoided is never looked at; or, if it is >looked at it, never looks attractive (in evaluation) to the engine. Obviously >such poor engines appear to be more like random movers, and the odds of avoiding >a normally attractive 'avoid move' would be high. > >Quantity forty (40) avoid move problems gives a good relative test among >programs. > >The relative test is not of overall program strength, Uri, but as others >indicated, only a test of how the program does on *that* type of problem test. > >If a known very weak engine got a perfect 'avoid move' test score, I would never >extrapolate (wrongly) from that result that the engine was the best at playing a >full game of chess. That would be ludicrous. > >The proper use of the test is to discriminate among programs to see their level >of ability to 'avoid' bad moves. Nothing more, nothing less. > >The probability that the winner of the 'avoid move' test is also the best chess >program also depends on many other important factors--such as the ability to >pick the 'best move' relatively more often than its competitors. > >Don't read into the 'avoid move' test more than it provides. More importantly, >don't assume that most readers will read into the test more than it provides. > >Regards, >--Steve I know that test suites do not give exact picture of the strength of the engines but I was not talking about a small difference and Mint is not an engine that plays random moves. A good avoid move test suite is a test suite of positions when at least materialistic programs always like the move to avoid at short depth. It is not the case with the swiss test. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.