Author: Kurt Utzinger
Date: 23:33:27 08/27/04
Go up one level in this thread
On August 27, 2004 at 19:54:28, Dan Honeycutt wrote:
>Does anyone have an opinion if it is better for an engine to play itself or
>other engines when trying to determine if a change is an improvement?
>
>I had two versions of my engine, old and new, identical except for one
>difference (new did checks in 1st ply of qsearch, but my question is general).
>I played old and new in a tourney against themselves and 3 other engines, 1
>stronger, 1 weaker and 1 about equal. After a week and a half and nearly 1000
>games I found new was better than old against all three of the other engines,
>but lost head-to-head against old. This can't be; error margin I figure. So I
>burned another week and a half of computer time and got the same result. Does
>this make sense? With results like this, how does one know if a change is any
>good?
>
>Any advice/opinions appreciated.
>Dan H.
Opinions differ, but from my own experience with testing
a lot of CM9-settings I can honestly say that it does not
make much (if at all) sense to test an engine to play
itself against other settings. The best way seems to test
in 30-50 games matches vs 10-20 other engines and at the
same time to avoid time controls 1-5m/g, instead 5m+2s at
least should be tried but better slower time controls.
Kurt [http://www.utzingerk.com]
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.