Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Bob, please forget WAC testsuite in Crafty testing!!

Author: Robert Hyatt

Date: 06:49:33 10/12/98

Go up one level in this thread


On October 12, 1998 at 01:04:34, Jouni Uski wrote:

>I see in in MAIN.C file, that version 15.20 has got 299/300 in WAC suite.
>I was very interested in testing it in ECM (from Fritz5) suite. And it was
>quite disappointing: only 63 solved in 1 minute run (P90)... Here is for
>comparison:
>
>Fritz4.01      120/216
>Fritz5         116
>Nimzo98        100
>Mchess7         89
>...
>Crafty 15.18    69
>Crafty 15.20    63 only...


Note...  I only use WAC to "validate" a version...  IE on occasion, I
will run WAC and get 293 or some such, which lets me know that something
has changed.  However, *my* test scores are going to go down I believe,
because solving test positions is not what I'm trying to do.  Crafty has
never been the best "problem solver" around, because I'm not willing to
waste a lot of time running checks out to 40 plies in the middlegame.  But
it does help to have WAC results for *every* version, then I can evaluate
changes to decide whether a little more depth in typical cases offsets the
reduced depth in other positions (like the WAC positions).

I don't understand your "ECM" test...  unless this is just selected
positions from ECM.  If you have just hard ECM positions, send them to
me if you can, I'll certainly look at them...



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.