Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: an idea to evaluate rating of programs based on pgn file of their games

Author: Bruce Moreland

Date: 09:56:59 08/09/01

Go up one level in this thread


On August 09, 2001 at 09:58:01, Robert Hyatt wrote:

>Very likely correct.  This is not an easy thing to do...  and trying to use
>program X to predict the rating of program Y, based only on how many moves they
>"match" looks statistically dangerous.
>
>This is essentially saying that DF played all the right moves, and any
>divergence by another program necessarily makes that program weaker.  IE in
>Uri's example, DB might appear worse that DF, because they didn't agree.  I'm
>certain it is not weaker even if they disagree 100% of the time.  Based on past
>games...

This is not quite true, I think.

He's saying that Fritz has X units of strength when it moves in Y seconds.  He
is concluding that at the point where another program has maximum agreement with
Fritz, it also has X units of strength.  If it gets more time than that, it will
start to disagree with Fritz, because it is making better moves.  With less
time, I will disagree because it is making worse moves.  Let's call the time of
maximum agreement Y'.  If Y' is less than Y, the new program is stronger, and it
may be possible to compute by how much, and vice versa.

I doubt this will work, as long as there are lots of positions where there are
more than one playable move.  You'd be looking for information in a big blob of
random soup.

bruce







This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.