Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Who is better? Some statistics...

Author: Gian-Carlo Pascutto

Date: 14:34:15 06/11/01

Go up one level in this thread


On June 11, 2001 at 16:58:11, David Rasmussen wrote:

>>>Certainly not that one is a better chess player than the other.
>>
>>Well, among those 2 engines the winner is the better chess
>>player. Unless you define 'better chess' player as something
>>else than that which will win the most games.
>>
>I do indeed define it as something else, when it comes to chess programs. The
>only way, for me at least, to determine which program is best, is to see how it
>scores against various opponents.  [notrelevant stuff snipped]

This is a different definition, but not one that invalidates mine,
since it applies to multiple programs, and mine applies to 2-program
matches. 'Various opponents' a priori invalidates applicability to
2 program matches.

>>It doesn't say a thing about performance vs something else,
>>but I find it interesting to compare different versions of
>>the same engine. If I fiddle and it suddenly plays significantly
>>better than the old version, that is a good indication that
>>that setting is better even vs other programs/humans.
>>
>No it isn't. It is wellknown that this type of selftesting leads to
>inbreeding, because both programs can't punish eachother for mistakes
>they don't understand.

I may pick up an improvement that actually leaves another hole,
but that will be caught when I do regression testing vs other
engines. Regression testing will also reveal improvements that
actually only punch holes into weak spots of my own (old) program.

However, an actual significant improvement should also improve the
score during the selfmatches. And so it will be caught.

What selfmatching will not catch are improvements which have the
potential to punch holes in weak spots of other programs. And I'm
not terribly interested in those.

--
GCP



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.