Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Bob, please forget WAC testsuite in Crafty testing!!

Author: Bernhard Bauer

Date: 23:23:34 10/12/98

Go up one level in this thread


On October 12, 1998 at 09:49:33, Robert Hyatt wrote:

>On October 12, 1998 at 01:04:34, Jouni Uski wrote:
>
>>I see in in MAIN.C file, that version 15.20 has got 299/300 in WAC suite.
>>I was very interested in testing it in ECM (from Fritz5) suite. And it was
>>quite disappointing: only 63 solved in 1 minute run (P90)... Here is for
>>comparison:
>>
>>Fritz4.01      120/216
>>Fritz5         116
>>Nimzo98        100
>>Mchess7         89
>>...
>>Crafty 15.18    69
>>Crafty 15.20    63 only...
>
>
>Note...  I only use WAC to "validate" a version...  IE on occasion, I
>will run WAC and get 293 or some such, which lets me know that something
>has changed.  However, *my* test scores are going to go down I believe,
>because solving test positions is not what I'm trying to do.  Crafty has
>never been the best "problem solver" around, because I'm not willing to
>waste a lot of time running checks out to 40 plies in the middlegame.  But
>it does help to have WAC results for *every* version, then I can evaluate
>changes to decide whether a little more depth in typical cases offsets the
>reduced depth in other positions (like the WAC positions).
>
>I don't understand your "ECM" test...  unless this is just selected
>positions from ECM.  If you have just hard ECM positions, send them to
>me if you can, I'll certainly look at them...

Hi,
here is the result for Crafty15.20 on a 2xPPro 233 MHz:

total acd: 2519   mean total acd: 11.66
total acn: -990304611   mean total acn: -4584743.57
total acs: 37850   mean total acs: 175.23
total mean node frequency: -26163.93 Hz
solve: 145   solve percent: 67.13
solve acd: 1711   mean solve acd: 11.80
solve acn: 824256592   mean solve acn: 5684528.22
solve acs: 24994   mean solve acs: 172.37
solve mean node frequency: 32978.18 Hz
unsol: 71   unsol percent: 32.87
unsol acd: 808   mean unsol acd: 11.38
unsol acn: -1814561203   mean unsol acn: -25557200.04
unsol acs: 12856   mean unsol acs: 181.07
unsol mean node frequency: -141145.08 Hz

which means that crafty solved 145 from 216 within 3 min.
Thats not too bad IMHO.

Kind regards
Bernhard



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.