Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Lightning ratings 30 programs 17,400 games

Author: David Dory

Date: 12:19:19 03/22/02

Go up one level in this thread


On March 22, 2002 at 14:09:30, K. Burcham wrote:

>Tiger and Shredder: getting conflicting tournament results of program strength.
>
>I know some of these are different game times. Blitz, lightning, standard, etc.
>I was comparing same game times. and please dont tell me you have to have play
>938.2 games to draw any conclusions.
>
>another personal conclusion: If I run a tournament and my results are very
>different than Sarah's----I will not post the results. I will look for software
>problems, ram problems with memory hogs, cpu problems with cpu hogs, ponder on
>issues, pc  connection issues, etc.
>kburcham

One key thing I've noticed. In a lot of these tournaments, the hash size is
***************** SET BY THE OPERATOR TO SOME ARBITRARY SIZE ***************
They say "Fair's fair"! But there's nothing to judge program strength about when
the operator interferes with the program's resources.

Why should Shredder with 32Meg hash table be the same strength as Shredder with
128Meg hash table?? Why should ANY program??

You'll always have differences in strength with the same program at different
speeds, of course. Some are better at blitz, others better at the longer time
controls.

I don't know what could really be fair, and show the programs at their full
strength except to let the programs select THEIR OWN hash table size.

Another trashmo idea - setting the program's opening book to something other
than it's own. How could anyone think that's fair??

Rather like cutting off someone's arm, and surgically attaching someone else's
arm to his body - and saying "Now we'll have a weightlifting contest!"

You're right to be suspicious of a lot of the results we see. IMO fair is fair,
but fair they ain't.

Dave








This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.