Author: Vasik Rajlich
Date: 08:26:28 04/13/04
Go up one level in this thread
On April 13, 2004 at 09:26:54, Vincent Diepeveen wrote: >On April 12, 2004 at 14:45:28, Christophe Theron wrote: > >>On April 12, 2004 at 07:50:47, Tord Romstad wrote: >> >>>On April 12, 2004 at 00:09:48, Christophe Theron wrote: >>> >>>>On April 11, 2004 at 13:52:59, Tom Likens wrote: >>>> >>>>>On April 10, 2004 at 21:53:17, Christophe Theron wrote: >>>>> >>>>>>On April 10, 2004 at 15:55:17, Tom Likens wrote: >>>>>>> >>>>>>>I'm not sure where I come down on the bitboards vs. non-bitboard >>>>>>>architectures. My engine is a bitboard engine, but that doesn't >>>>>>>necessarily mean that the next one will be bitboard based. >>>>>>> >>>>>>>I don't believe though, that because no bitboarder has topped the >>>>>>>SSDF list that this really constitutes any kind of proof. My strong >>>>>>>suspicion is that if all the top commercial programmers converted >>>>>>>over to bitboards tomorrow (yourself included) that *eventually* >>>>>>>their new engines would again rise to the top of the SSDF. I'm >>>>>>>beginning to suspect that creating a strong (i.e. world-class) engine >>>>>>>involves a helluva lot more than just the basic data representation, >>>>>>>but instead involves... >>>>>>> >>>>>>>1. 24/7 dedication >>>>>>>2. A *real* way to measure progress >>>>>>>3. A selective search strategy that works 99.99999% of the time >>>>>>>4. Attention to about 2^64 minor details >>>>>>>5. A failed marriage (okay, maybe this is extreme but you see the point) >>>>>>> >>>>>>>regards, >>>>>>>--tom >>>>>> >>>>>> >>>>>> >>>>>>Number 5 (or something close) was the reason why Tiger has made such a progress >>>>>>between 1997 and 1999. :) >>>>>> >>>>>>Number 2, seriously, is worth spending several months on it. >>>>>> >>>>>> >>>>>> >>>>>> Christophe >>>>> >>>>>This has been my main focus over the past few weeks. It's become readily >>>>>apparent to me that the improvement slope from here on up is much steeper >>>>>and I rather not waste my time implementing features that I can't properly >>>>>test. >>>>> >>>>>regards, >>>>>--tom >>>> >>>> >>>> >>>>That's the secret of real professional chess programmers. >>> >>>Of course you don't want to reveal your secrets, but it would be interesting if >>>you could answer >>>the following question: >>> >>>Assume that you make a change to your engine which improves the playing strength >>>by >>>about 10 Elo points. How many hours of CPU time do you need before you are sure >>>that >>>the change was an improvement? >>> >>>Tord >> >> >> >>I would say approximately one week, and I would not even be really sure it is an >>improvement. We are talking about a 1.5% improvement in winning percentage here, > >That's very quick. I remember 1 test of diep which took 3 months to get tested >by Jan Louwman. 1000 games in total. About 500 for version A and about 500 for >version B. Level 40 in 2. > >>it's below the statistical noise of a several hundreds games match if you want >>95% reliability! > >>And unfortunately a 10 elo points improvement is becoming rare for me. Most of >>the changes I try make the program weaker, and many changes do not provide any >>measurable improvement! > >Another sneaky problem is that if you try to modify some behaviours of your >program, it first will score less until everything is retuned. > >>That's why not having a strong test methodology is totally out of question if >>you are serious about chess programming. > >I agree here with you. > >At the same time mentionning that a good test methodology has its limitations >too. If you just believe the tests without taking into account other factors >then it is a wrong concept too. > >Note that diep can use some testing, nothing is done systematic currently and >the number of testgames can be counted at 1 hand a day, if there is anything to >report at all ;) > >>Even with a good test methodology chess programming is still an art: in many >>cases you have to decide with your feelings, because the raw data does not give >>you a definite answer. >> >>Now of course there are small improvements that I do not even need to test for a >>long time: if I find a way to make my program 10% faster without changing the >>shape of the tree, then all I need to do is run some safety tests that will only >>look at the number of nodes searched on a large set of positions and compare it >>to the last stable version. > >Let's discuss that "without changing the shape of the tree". I suspect you mean >by that forward pruning. > >So far tests have convincingly showed (take the 1000 games of Louwman) that not >a single forward pruning experiment tried so far for DIEP has resulted in better >play. > Vincent, That's a great experiment. I'm curious, have you tried the same thing for your beloved evaluation? Ie. rip out enough to run let's say half of fritz speed, and self-play 1000 games? Vas >There is 1 week to go to now to the ict4 and i see that i can possibly win 1500 >dollars. Now the problem is this time no good hardware for DIEP so i'll have to >find some spectacular improvements in DIEP in order to win that price :) > >Now i decided that forward pruning in the entire tree (i do not see nullmove as >a problem by the way) that so far all those experiments failed for DIEP. > >Plydepthleft 3 to 4 in DIEP i do next type of nullmove: > > score = -qsearch( .. ); > > if( score >= beta ) return score; > >Of course as already explained it's not implemented like this because i'm >non-recursive, but it very well describes the rudeness in which i do things. > >Measuring my tree i see that the vaste majority of nodes searched by diep are >qsearch nodes. About 80% of all nodes are in qsearch. > >I'm interested now in redoing a forward pruning experiment. > >The current thought is to forward prune at the last ply. So i try to select a >few moves there and play them. > >Now of course the amount of pruning is dependant upon how many moves you try >there. > >Yet i'm wondering what i can potentially win in search depth just weeding in >that last ply. > >What is your estimate? > >>But that does not happen often! :) >> >> >> >> Christophe
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.