Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Engine ratings vs humans: experiment design

Author: David Dory

Date: 06:46:26 12/26/01

Go up one level in this thread


On December 25, 2001 at 09:40:03, Edward Seid wrote:

>- each engine will have its blitz rating initialized to an estimated strength
I believe more valid results would be obtained by letting the actual results set
this rating. The less you "do" to bias the outcome in any way, the better.

>My questions are the following:
>
>- time control: 5 12 also being considered.  pros/cons?
>- hardware: also have weaker PII/233 with 128 MB RAM running Windows 98
>available.  pros/cons to running experiment with weaker hardware?
I'd keep it on the stronger hardware. A PII/233 would only cripple good play
especially in a blitz time control.

>- hashtable size: deciding between 64, 96 or 128 MB
Perhaps the engines could just use their default setting (which should be their
strongest, we hope)?

>- hashtable allocation: if configurable, how to allocate total hash between
>hashsize, q_hashsize, pawn_hashsize?
Again, go with default. You can only be influencing the programs performance
when you start tweaking with the settings. When the program then wins or loses,
it will be as much because of your tweaking as because of the programs strength.


>- EGTBs: currently deciding between 3/4 complete or 3/4/5 complete.  pros/cons?
Doesn't matter, we're not solving problems, this is sheer competition, so just
make it equal for all programs.

>- resign value: trying to decide between -7 and -9. comments?
Minus 7 should be fine, and save a lot of time.

Dave



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.