Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Engine testing: Who do you play?

Author: Robert Hyatt

Date: 13:40:04 09/01/04

Go up one level in this thread


On August 31, 2004 at 14:30:21, Chris Welty wrote:

>On August 31, 2004 at 10:28:04, Robert Hyatt wrote:
>
>>On August 31, 2004 at 08:22:20, Chris Welty wrote:
>>
>>>On August 30, 2004 at 17:01:57, Robert Hyatt wrote:
>>><snip>
>>>>>Auto-tuning would be great.  My problem is I don't know how to do it.  If you
>>>>>have a method I'd love to hear more.
>>>>>
>>>>>Dan H.
>>>>
>>>>
>>>>It is also a _very_ hard problem.
>>>>
>>>>_very_ hard...
>>>
>>>It was quite easy to get Altamax to autotune its evaluation parameters so that
>>>it played substantially better than the hand-tuned version. It took less time
>>>than hand-tuning too.
>>>
>>>Getting autotuning to work WELL required some work, but was still easier than
>>>hand-tuning the coefficients.
>>>
>>>I use TD-leaf, which I got from some paper on the internet.
>>>
>>>Chris
>>
>>
>>How many different terms?  Crafty has a bunch.  auto-tuning them to be better
>>than the hand-tuned default has not happened yet although we have (mainly
>>Anthony) beat on it mercilously. Crafty has about 120 different eval features
>>that can be adjusted, with maybe 1/2 of them represented as large arrays of
>>values.  Easily over 1000 different values to adjust.  A tough task as I said...
>
>When I did the initial comparison, about 75 terms. Since autotuning is easy I've
>added lookup tables; now there are a total of 570 numbers that can be adjusted.
>
>The piece-square table numbers do look strange (-0.95 pawns for a bishop on
>D8??) but the results are good.

That has to be the result of a poor test suite.  IE with a score like that, the
game will revolve around one program giving up a pawn to make the other program
move its bishop to D8, which makes no sense in real chess...


>
>I suspect it also depends on how good you are at hand tuning.



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.