Computer Chess Club Archives




Subject: Re: Book learning and rating bias

Author: Thorsten Czub

Date: 17:10:46 05/04/98

Go up one level in this thread

On May 01, 1998 at 13:58:29, Don Dailey wrote:
>One very important factor is book learning and I do  not know how this
>is  handled  by the raters, hopefully   it  is handled correctly.  The
>issue is that if  I  have a program  that  learns from it's   mistakes
>(which I think  is a very good thing,)  then that program should never
>be "reset" by the testing procedure.  As an example, if I was a biased
>tester,  I could simply   reset the learning mechanism frequently  and
>affect the results (perhaps) significantly.  I  might move the program
>from machine to  machine or whatever it takes  to defeat the  learning

If Fritz5 gets only ONE machine (since it needs special autoplayer,
special hash-ram and big HD (copying the 550 MByte book ...) it has
Hiarcs and Rebel and Mchess can be installed almost anywhere.
And so this implies they would get many testers and many machines.

>Having several testers testing  the same program on different machines
>creates the same problem.


> I argue that the  more computers you use to
>test a program on, the more of a handicap  you give to that program if
>it utilizes  learning mechanisms.  I  don't know  the magnitude of the
>error but it  certainly would  be a  factor   to  consider.  The  only
>solution I am aware of is to use the  same machine to test the program
>on.   If  you  use other  machines  you  must consider   them separate

This page took 0.01 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.