Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Test Suites -- How do you decide on pass or fail?

Author: Adrien Regimbald

Date: 04:08:21 06/28/00

Go up one level in this thread


Hi,

>I am just adding code to GreenLight to run EPD test suites... but I have hit a
>problem.
>
>Just how do you mark the results?!
>
>At the moment I am doing this:
>Take static eval of start position.
>Start engine thinking.
>If the correct move is seen with (eval >= start_eval + 1.00) then say passed.
>If after max. time, move is not taken -> mark as failed, else mark as uncertain.
>
>I guess I could just go for the criteria that if 3 plys in a row see the correct
>move, then it passes?  Does this work?
>
>Any suggestions welcome.


Most of the schemes I've seen do something extremely simple:

The user specifies somehow how much time will be spent on each problem.  If the
move that the engine thinks is best at the end of that time is the correct move,
it passed.  If the program thought the "avoid move" was best, usually this is
flagged for further investigation, as that move should really REALLY not be
played :P


Regards,
Adrien.



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.