Author: Bill McGaugh
Date: 21:03:51 02/11/99
Go up one level in this thread
Paulo, Yes...the idea is that a program "plays a tournament" against the previously rated (or calibrated) problems...and receives a performance rating based on its win-loss record. The method is not without problems...ideally the problems would be calibrated using a wide range of programs...and the programs would be rated using a very large number of these calibrated problems... My initial efforts are based on far too few programs for problem ratings. The error of the measurement is probably somewhat different from Elo's formulae, since the initial problem ratings, which the program ratings are based on, are established from the problem performance of a small number of programs. An advantage of this method is that, after you calibrate problems, you can pick the ones you want for a test suite...and test the performance of the new suite...the real test being how well a suite estimates the performance of programs that are completely unrelated to the calibration of the suite... This method is a lot of fun to play with, but it is probably a pursuit of the impossible dream...a set of problems that will quickly rate chess programs...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.