Author: Christophe Theron
Date: 11:45:28 04/12/04
Go up one level in this thread
On April 12, 2004 at 07:50:47, Tord Romstad wrote:
>On April 12, 2004 at 00:09:48, Christophe Theron wrote:
>
>>On April 11, 2004 at 13:52:59, Tom Likens wrote:
>>
>>>On April 10, 2004 at 21:53:17, Christophe Theron wrote:
>>>
>>>>On April 10, 2004 at 15:55:17, Tom Likens wrote:
>>>>>
>>>>>I'm not sure where I come down on the bitboards vs. non-bitboard
>>>>>architectures. My engine is a bitboard engine, but that doesn't
>>>>>necessarily mean that the next one will be bitboard based.
>>>>>
>>>>>I don't believe though, that because no bitboarder has topped the
>>>>>SSDF list that this really constitutes any kind of proof. My strong
>>>>>suspicion is that if all the top commercial programmers converted
>>>>>over to bitboards tomorrow (yourself included) that *eventually*
>>>>>their new engines would again rise to the top of the SSDF. I'm
>>>>>beginning to suspect that creating a strong (i.e. world-class) engine
>>>>>involves a helluva lot more than just the basic data representation,
>>>>>but instead involves...
>>>>>
>>>>>1. 24/7 dedication
>>>>>2. A *real* way to measure progress
>>>>>3. A selective search strategy that works 99.99999% of the time
>>>>>4. Attention to about 2^64 minor details
>>>>>5. A failed marriage (okay, maybe this is extreme but you see the point)
>>>>>
>>>>>regards,
>>>>>--tom
>>>>
>>>>
>>>>
>>>>Number 5 (or something close) was the reason why Tiger has made such a progress
>>>>between 1997 and 1999. :)
>>>>
>>>>Number 2, seriously, is worth spending several months on it.
>>>>
>>>>
>>>>
>>>> Christophe
>>>
>>>This has been my main focus over the past few weeks. It's become readily
>>>apparent to me that the improvement slope from here on up is much steeper
>>>and I rather not waste my time implementing features that I can't properly
>>>test.
>>>
>>>regards,
>>>--tom
>>
>>
>>
>>That's the secret of real professional chess programmers.
>
>Of course you don't want to reveal your secrets, but it would be interesting if
>you could answer
>the following question:
>
>Assume that you make a change to your engine which improves the playing strength
>by
>about 10 Elo points. How many hours of CPU time do you need before you are sure
>that
>the change was an improvement?
>
>Tord
I would say approximately one week, and I would not even be really sure it is an
improvement. We are talking about a 1.5% improvement in winning percentage here,
it's below the statistical noise of a several hundreds games match if you want
95% reliability!
And unfortunately a 10 elo points improvement is becoming rare for me. Most of
the changes I try make the program weaker, and many changes do not provide any
measurable improvement!
That's why not having a strong test methodology is totally out of question if
you are serious about chess programming.
Even with a good test methodology chess programming is still an art: in many
cases you have to decide with your feelings, because the raw data does not give
you a definite answer.
Now of course there are small improvements that I do not even need to test for a
long time: if I find a way to make my program 10% faster without changing the
shape of the tree, then all I need to do is run some safety tests that will only
look at the number of nodes searched on a large set of positions and compare it
to the last stable version.
But that does not happen often! :)
Christophe
This page took 0.02 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.