Computer Chess Club Archives




Subject: Re: Value of playing different versions of a program against each other

Author: Peter Fendrich

Date: 14:30:00 01/06/03

Go up one level in this thread

On January 06, 2003 at 16:56:35, Tom King wrote:

>Hi all,
>What do people think about playing different versions of your program against
>each other as a way of testing?
>I'm playing around with it right now, between v0.07 and a newer version of my
>program. The newer version is winning handsomely: +24,=18,-10.
>This implies a reasonably impressive increase in strength, almost 100 ELO. Ok,
>ok, it's a small sample, so the margin of error could be big.
>However, my gut feel is that playing different versions of your programs tends
>to overstate the strength differences. What do people think?

I wouldn't rely on figures based only on games against a previous version.
However, I think it's a good way to get a hint if a certain change in your
program qualifies for a broader test.
In general, if your new version doesn't beat your previous one significantly
it's probably not worth the effort to go on testing that particular change.

I'm talking about incremental changes not a total rewrite...


This page took 0.01 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.