Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Test Suites

Author: Michael Yee

Date: 18:32:06 12/08/03

Go up one level in this thread


On December 08, 2003 at 20:38:03, Russell Reagan wrote:

>On December 08, 2003 at 20:27:33, Steven Edwards wrote:
>
>I like the NATS idea.
>
>>One plan is to categorize position difficulty based on
>>
>>    d = log N     d: difficulty    N: node count to solution
>
>Would the method for counting nodes be standardized? Some count nodes
>differently than others. For instance, one might not count the nodes visited
>during a null-move search and instead count the root node from which that
>null-move search was performed as a single node. Some programs do a great deal
>more work per node. Maybe more than one metric should be used. I think time is a
>better one than nodes. Nodes seem dependent upon the design philosophy of the
>engine writer.
>
>The times would go down as hardware gets faster, but that is part of the point
>of test suites, to determine how computer chess has advanced, and hardware is a
>part of that equation.

You can normalize the times to sum to 1 so that scaling the hardward has no
effect. For instance, solution times of 1, 2, and 8 would become 1/10, 2/10, and
8/10. Then if the hardware became 4 times faster, the normalized values would be
4/40, 8/40, and 32/40 (which are the same as before).

I agree that using time would take care of different definitions of "node".

Michael



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.