Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Don't trust testsuites too much: one good example

Author: Dusan Dobes

Date: 02:13:54 02/18/99

Go up one level in this thread


On February 18, 1999 at 04:24:18, Jouni Uski wrote:

>Fritz5 ECM is tactical testsuite with a lot of sacrifices in 216 positions.
>Phalanx XX solves full 100 positions in 1 min level in P90. On contrary Crafty
>16.2 solves "only" 73. But in actual play Phalanx loses almost all games against
>Crafty! Score in 5 min blitz Phalanx - Crafty 3 - 17...

Phalanx has quite large king safety bonuses in it's static evaluation.
These bonuses make it often sacrifice material for king safety.
In practical play, these sacrifices are sometimes correct, sometimes
incorrect (but always funny and even incorrect sacrifices often win
against humans). In test suites, the sacrifices are always correct,
that's the way how the positions in test suites are selected.

I agree.  Running test suites is a poor way to compare the playing
strength.  Running comp-comp blitz matches isn't much better either.
I admit that i run these matches too :-).  I use longer time controls
(e.g. 30 minutes + 30s increment on a P90).  My results against
Crafty are not that bad.

I suggest upgrading Phalanx to version XXI.  Phalanx XX has an ugly
bug in the static evaluation that make it drop all it's material
in most engames with outside passed pawns.  It was detected by
Zeke Smigel who was running Phalanx on ICC, this is the critical
position: 8/6k1/6b1/3P2P1/6R1/2N1q3/1P3RK1/8 w - - 0 51  Rf7+??

Dusan



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.