Author: Vincent Diepeveen
Date: 06:26:54 04/13/04
Go up one level in this thread
On April 12, 2004 at 14:45:28, Christophe Theron wrote: >On April 12, 2004 at 07:50:47, Tord Romstad wrote: > >>On April 12, 2004 at 00:09:48, Christophe Theron wrote: >> >>>On April 11, 2004 at 13:52:59, Tom Likens wrote: >>> >>>>On April 10, 2004 at 21:53:17, Christophe Theron wrote: >>>> >>>>>On April 10, 2004 at 15:55:17, Tom Likens wrote: >>>>>> >>>>>>I'm not sure where I come down on the bitboards vs. non-bitboard >>>>>>architectures. My engine is a bitboard engine, but that doesn't >>>>>>necessarily mean that the next one will be bitboard based. >>>>>> >>>>>>I don't believe though, that because no bitboarder has topped the >>>>>>SSDF list that this really constitutes any kind of proof. My strong >>>>>>suspicion is that if all the top commercial programmers converted >>>>>>over to bitboards tomorrow (yourself included) that *eventually* >>>>>>their new engines would again rise to the top of the SSDF. I'm >>>>>>beginning to suspect that creating a strong (i.e. world-class) engine >>>>>>involves a helluva lot more than just the basic data representation, >>>>>>but instead involves... >>>>>> >>>>>>1. 24/7 dedication >>>>>>2. A *real* way to measure progress >>>>>>3. A selective search strategy that works 99.99999% of the time >>>>>>4. Attention to about 2^64 minor details >>>>>>5. A failed marriage (okay, maybe this is extreme but you see the point) >>>>>> >>>>>>regards, >>>>>>--tom >>>>> >>>>> >>>>> >>>>>Number 5 (or something close) was the reason why Tiger has made such a progress >>>>>between 1997 and 1999. :) >>>>> >>>>>Number 2, seriously, is worth spending several months on it. >>>>> >>>>> >>>>> >>>>> Christophe >>>> >>>>This has been my main focus over the past few weeks. It's become readily >>>>apparent to me that the improvement slope from here on up is much steeper >>>>and I rather not waste my time implementing features that I can't properly >>>>test. >>>> >>>>regards, >>>>--tom >>> >>> >>> >>>That's the secret of real professional chess programmers. >> >>Of course you don't want to reveal your secrets, but it would be interesting if >>you could answer >>the following question: >> >>Assume that you make a change to your engine which improves the playing strength >>by >>about 10 Elo points. How many hours of CPU time do you need before you are sure >>that >>the change was an improvement? >> >>Tord > > > >I would say approximately one week, and I would not even be really sure it is an >improvement. We are talking about a 1.5% improvement in winning percentage here, That's very quick. I remember 1 test of diep which took 3 months to get tested by Jan Louwman. 1000 games in total. About 500 for version A and about 500 for version B. Level 40 in 2. >it's below the statistical noise of a several hundreds games match if you want >95% reliability! >And unfortunately a 10 elo points improvement is becoming rare for me. Most of >the changes I try make the program weaker, and many changes do not provide any >measurable improvement! Another sneaky problem is that if you try to modify some behaviours of your program, it first will score less until everything is retuned. >That's why not having a strong test methodology is totally out of question if >you are serious about chess programming. I agree here with you. At the same time mentionning that a good test methodology has its limitations too. If you just believe the tests without taking into account other factors then it is a wrong concept too. Note that diep can use some testing, nothing is done systematic currently and the number of testgames can be counted at 1 hand a day, if there is anything to report at all ;) >Even with a good test methodology chess programming is still an art: in many >cases you have to decide with your feelings, because the raw data does not give >you a definite answer. > >Now of course there are small improvements that I do not even need to test for a >long time: if I find a way to make my program 10% faster without changing the >shape of the tree, then all I need to do is run some safety tests that will only >look at the number of nodes searched on a large set of positions and compare it >to the last stable version. Let's discuss that "without changing the shape of the tree". I suspect you mean by that forward pruning. So far tests have convincingly showed (take the 1000 games of Louwman) that not a single forward pruning experiment tried so far for DIEP has resulted in better play. There is 1 week to go to now to the ict4 and i see that i can possibly win 1500 dollars. Now the problem is this time no good hardware for DIEP so i'll have to find some spectacular improvements in DIEP in order to win that price :) Now i decided that forward pruning in the entire tree (i do not see nullmove as a problem by the way) that so far all those experiments failed for DIEP. Plydepthleft 3 to 4 in DIEP i do next type of nullmove: score = -qsearch( .. ); if( score >= beta ) return score; Of course as already explained it's not implemented like this because i'm non-recursive, but it very well describes the rudeness in which i do things. Measuring my tree i see that the vaste majority of nodes searched by diep are qsearch nodes. About 80% of all nodes are in qsearch. I'm interested now in redoing a forward pruning experiment. The current thought is to forward prune at the last ply. So i try to select a few moves there and play them. Now of course the amount of pruning is dependant upon how many moves you try there. Yet i'm wondering what i can potentially win in search depth just weeding in that last ply. What is your estimate? >But that does not happen often! :) > > > > Christophe
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.