Author: David Dahlem
Date: 09:10:57 06/20/04
Go up one level in this thread
On June 19, 2004 at 12:02:19, Steve Glanzfeld wrote: >On June 19, 2004 at 11:47:03, Gian-Carlo Pascutto wrote: > >>On June 19, 2004 at 11:41:04, Steve Glanzfeld wrote: > >>>http://www.talkchess.com/forums/1/message.html?371210 >>> >>>Doing something beats blah. :) >> >>I can understand some users might have some irresistable desire to >>assign "ratings" via "testsets", but that does not change the fact that >>this procedure is meaningless, unreliable, and a waste of time. > >Users are interested in the engine's performances in such tests, simply. >Assingning ratings certainly isn't the main thing. Most often it is sufficient >to count and compare the number of solutions. > I've seen numerous examples of one engine solving a test suite position in a few seconds, while another engine of known equal game playing strength never finds the solution, even after hours of analysis. To me, this makes test suites worthless, or at least very difficult to interpret the results. Regards Dave >A practical example: A chessplayer has trouble in minor piece endings. He wants >to analyse his games where such positions occured, with a chess program and >maybe use that same program for training, to learn to handle these positions >better. Now, he'd like to choose an engine which is especially strong in that >type of positions, from several generally strong engines he has. So he compiles >a test from typical minor piece ending positions (maybe from books, GM >analysis...) where there is a specific difficult good move to be found, or a >good looking but in fact bad move to be avoided (= solve). And then, he runs >that test with the engines he has available, unsing roughly the time per >position he intend to use in analysis. The engine which solves most often, is >now his preferred analysis engine for minor piece endings. > >Now please don't tell me that wouldn't be ok... > >So what's all that "flawed" (etc.) blah?? > >Steve
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.