Author: Howard Exner
Date: 18:41:51 07/25/98
Go up one level in this thread
On July 25, 1998 at 19:00:53, Mark Gaddis wrote: >Does anyone know of a set of chess positions which can be used to check both the >tactical and positional prowess of a chess program? There are many test suites available on various sites. One site is Shep's Chess Homepage at http://www.mi.uni-koeln.de/~kluebke/chess/compchess.html Another one is ECM98, a modified version of "Encyclopedia of Chess Middlegames". It was this group here on CCC who modified it. It is found at Manfred Rosenboom's Chess site http://ourworld.compuserve.com/homepages/Manfred_Rosenboom/chess.htm There are various opinions on how well such test suites can gage the strength of programs. I'm of the opinion that suites are not a good predicter in determining a rating. Some programmers have shared here that they developed a version of their program that overall did better on test suites but that overall played weaker chess. I'm not a programmer but there must be some kind of a trade off when it comes to this sort of work in improving a program. Programmers, does this happen - You've got your program to play certain types of positions better but now they are doing something else that they shouldn't be? Another question for programmers. Do you come across certain positions and say, "I would like my program to play that move - now what must I change in order to make that happen?" >It might be possible to get >an approximate rating for a program by analysing the percentage of correct >solutions determined and the time it took to arrive at these solutions. This >would save a tremendous amount of time versus playing many actual games against >a rated opponent. I think we are stuck with relying on humans for a true rating. We have to go through that process of playing 25 people to get an established rating. Computers must do the same, but it seems a difficult task to arrange the 40/2 matches. I'm always wishing for a sponsor like Intel or AMD to swoop down and fund regular Human vs Machine events, reasoning that they would get some publicity back in return. It would be nice if such a test suite where available but it would not be enough to cover the broad scope chess offers. Besides, if a test suite could accurately predict a rating it would ruin all our fun here at CCC in debating how strong these current programs are becoming.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.