Author: Mike Byrne
Date: 09:11:51 06/19/04
Go up one level in this thread
On June 19, 2004 at 12:02:19, Steve Glanzfeld wrote: >On June 19, 2004 at 11:47:03, Gian-Carlo Pascutto wrote: > >>On June 19, 2004 at 11:41:04, Steve Glanzfeld wrote: > >>>http://www.talkchess.com/forums/1/message.html?371210 >>> >>>Doing something beats blah. :) >> >>I can understand some users might have some irresistable desire to >>assign "ratings" via "testsets", but that does not change the fact that >>this procedure is meaningless, unreliable, and a waste of time. > >Users are interested in the engine's performances in such tests, simply. >Assingning ratings certainly isn't the main thing. Most often it is sufficient >to count and compare the number of solutions. > >A practical example: A chessplayer has trouble in minor piece endings. He wants >to analyse his games where such positions occured, with a chess program and >maybe use that same program for training, to learn to handle these positions >better. Now, he'd like to choose an engine which is especially strong in that >type of positions, from several generally strong engines he has. So he compiles >a test from typical minor piece ending positions (maybe from books, GM >analysis...) where there is a specific difficult good move to be found, or a >good looking but in fact bad move to be avoided (= solve). And then, he runs >that test with the engines he has available, unsing roughly the time per >position he intend to use in analysis. The engine which solves most often, is >now his preferred analysis engine for minor piece endings. > >Now please don't tell me that wouldn't be ok... > >So what's all that "flawed" (etc.) blah?? > >Steve I think test positions are great. They are a lot fun to see how engines stack against each other on the same position or suite of positions. But from my own experience in modifying Crafty , the best Crafty mods for test positions are not the same mods for playing better chess. However if I really screw up Crafty from playing good chess - I can usually tell by runnning a test suite. So I use it as a sanity check that I did not break Crafty. From my perspscetive , the programs that do best in test positions are simply the best programs that do best in test positions - nothing more , nothing less and it can be a lot of fun and interesting to some people. Other people may not find it fun and interesting.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.