Computer Chess Club Archives




Subject: Re: Testing the reliability of forward pruning

Author: Uri Blass

Date: 02:06:46 05/16/03

Go up one level in this thread

On May 15, 2003 at 22:28:17, Ross Boyd wrote:

>On May 15, 2003 at 21:47:20, Jon Dart wrote:
>>Ernst Heinz did this by using standard test suites, for example Win at Chess, or
>>ECM (Encyclopedia of Chess Middlegames). He found that the solve rate didn't
>>really change much with forward pruning on, but the number of nodes searched for
>>a fixed ply depth decreased 20-50% (this is from the chapter on AEL pruning in
>>his book Scaleable Search in Computer Chess). He also used other testing
>>methods, including game play, as detailed in the book.
>I use WAC to measure gains in performance but there is a real danger in this.
>Once as an experiment I modified my qsearch to radically prune 'losing' captures
>. This resulted in a huge improvement to my WAC scores (from 280 to 292 solved).
>But my engine played much much worse (than it already did) over the board.
>So, I concur with Jon (and Ernst, I guess) that gameplay is vital to gauge true
>performance. I would guess there is no quick way to prove the worth of an
>algorithm change. Obviously test suites can alert you to obvious problem areas
>but should be treated with some suspicion.

Maybe WAC is not the right test suite to try.

I think that a good test suite should be based on tactical errors of computers
in games when the target is to avoid them.

I was too lazy to build that test suite until today.


This page took 0.01 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.