Author: Tim Mirabile
Date: 09:06:37 02/12/99
Go up one level in this thread
On February 11, 1999 at 15:41:25, Dann Corbit wrote: >Andreas Schwartmann asked an interesting question in r.g.c.c.: >"I wonder if anyone can enlighten me on how to use various test suites, like >LCT, LCT II and Covax. There are ceratin formulas on how to calculate the >playing strength according to these test suites, right?" > >Now, ignoring the fact that they are full of bugs and the measures are probably >bogus, how *does* one arrive at an ELO from a test suite evaluation? > >What is the actual mathematical basis for the calculations? I don't know if anyone does anything like this now, but what I would do is this: - Give the test suite to a large number of human players of a wide variety of strengths and who have fairly stable ratings. - Find a weight for each problem based upon who solves it, who doesn't, and possibly the solving time. - Create a scoring formula empirically from this data which takes into account not only how many problems were solved, but which ones, and possibly how long it took to solve each one. This runs into the same practical problem as trying to find enough high rated humans to simply play rated games against the various programs, but at least here it's not so bad - once the test is calibrated it can be used on any program with no further need for the rated human players.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.