Author: Steven Edwards
Date: 17:27:33 12/08/03
Go up one level in this thread
On December 08, 2003 at 19:55:02, Russell Reagan wrote:
>Are there already test suites in which the majority of positions are difficult
>for computers? I only know of the Nolot suite.
There are a number of suites whose contents have been entirely culled from other
suites, but it is unclear if any have had a formal review.
One of my ideas for the NACCA is to have an official NATS.epd (NA Test Suite)
with a thousand positions selected from GM games and tested by the NACCA
membership. Each year, the NATS would be revised with the previous year's
version archived for long term comparison analysis. So if someone wrote a paper
referencing NATS/2004, the results would still be meaningful years later.
One plan is to categorize position difficulty based on
d = log N d: difficulty N: node count to solution
using the natural logarithm base.
Contributing programs will have different values for d, but these can be
normalized and averaged to construct a nominal d for each position. The NATS
positions are then ordered by difficulty, easy to hard.
The result is of most use to active developers. A thousand positions should
have a good overall coverage, and an ongoing engine development process could
run the NATS a hundred consecutive positions at a time. Once a sufficient
percentage were solved, the effort could move on to the next group. Using the
traditional three minutes per move average, a hundred element subset could be
started a bit before bedtime and be ready a bit after breakfast.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.