Author: Charles Roberson
Date: 19:39:12 01/22/03
Go up one level in this thread
Your idea has merit.
All programs play all programs and new versions are not weighted by the
rating of old versions. So, basically a new version replaces the old one. And
you run a gauntlet for each new release of each new program. This produces a
resonable level of accuracy. A few lists are/were this way. But an arbitrary
central point of tendancy was used. You could improve this if you could rate
your "benchmark" programs.
The SSDF list is interesting but it is not for all programs.
I suggest WTBM to run gauntlets. Easy to use and well done.
For your seeded programs: I suggest taking a program like crafty and taking
out the qsearch and limiting the depth. Limiting the depth is done in the config
file -- so no recoding. Taking out the qsearch should be easy. From crafty you
could create several crippled versions very quickly.
Also, some data was publish about ratings for crippled craftys running on one
of the chess servers. This could be used a starting point. If I recall
correctly, crafty (some version 4 years ago) fluctuated between 1500 & 1600 with
just one ply of search (qsearch turned on).
Other issues:
o Hardware makes a difference
o Time controls make a difference
o This is time consuming -- over 150 programs and growing. At 150
programs and G/5 time contol and 2 games per match, you need a
maximum of 24.83 hours. This will probably average 18 hours. However,
you could use a 1.2 GHz machine and cut the time by 3x and claim a
rating for a 3x slower machine. Also, with a dual processor machine
and no pondering, you can cut the time in half by running two matches
at once.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.