Author: Vasik Rajlich
Date: 08:24:22 04/13/04
Go up one level in this thread
On April 13, 2004 at 09:40:41, Omid David Tabibi wrote: >On April 12, 2004 at 14:45:28, Christophe Theron wrote: > >>On April 12, 2004 at 07:50:47, Tord Romstad wrote: >> >>>On April 12, 2004 at 00:09:48, Christophe Theron wrote: >>> >>>>On April 11, 2004 at 13:52:59, Tom Likens wrote: >>>> >>>>>On April 10, 2004 at 21:53:17, Christophe Theron wrote: >>>>> >>>>>>On April 10, 2004 at 15:55:17, Tom Likens wrote: >>>>>>> >>>>>>>I'm not sure where I come down on the bitboards vs. non-bitboard >>>>>>>architectures. My engine is a bitboard engine, but that doesn't >>>>>>>necessarily mean that the next one will be bitboard based. >>>>>>> >>>>>>>I don't believe though, that because no bitboarder has topped the >>>>>>>SSDF list that this really constitutes any kind of proof. My strong >>>>>>>suspicion is that if all the top commercial programmers converted >>>>>>>over to bitboards tomorrow (yourself included) that *eventually* >>>>>>>their new engines would again rise to the top of the SSDF. I'm >>>>>>>beginning to suspect that creating a strong (i.e. world-class) engine >>>>>>>involves a helluva lot more than just the basic data representation, >>>>>>>but instead involves... >>>>>>> >>>>>>>1. 24/7 dedication >>>>>>>2. A *real* way to measure progress >>>>>>>3. A selective search strategy that works 99.99999% of the time >>>>>>>4. Attention to about 2^64 minor details >>>>>>>5. A failed marriage (okay, maybe this is extreme but you see the point) >>>>>>> >>>>>>>regards, >>>>>>>--tom >>>>>> >>>>>> >>>>>> >>>>>>Number 5 (or something close) was the reason why Tiger has made such a progress >>>>>>between 1997 and 1999. :) >>>>>> >>>>>>Number 2, seriously, is worth spending several months on it. >>>>>> >>>>>> >>>>>> >>>>>> Christophe >>>>> >>>>>This has been my main focus over the past few weeks. It's become readily >>>>>apparent to me that the improvement slope from here on up is much steeper >>>>>and I rather not waste my time implementing features that I can't properly >>>>>test. >>>>> >>>>>regards, >>>>>--tom >>>> >>>> >>>> >>>>That's the secret of real professional chess programmers. >>> >>>Of course you don't want to reveal your secrets, but it would be interesting if >>>you could answer >>>the following question: >>> >>>Assume that you make a change to your engine which improves the playing strength >>>by >>>about 10 Elo points. How many hours of CPU time do you need before you are sure >>>that >>>the change was an improvement? >>> >>>Tord >> >> >> >>I would say approximately one week, and I would not even be really sure it is an >>improvement. We are talking about a 1.5% improvement in winning percentage here, >>it's below the statistical noise of a several hundreds games match if you want >>95% reliability! >> >>And unfortunately a 10 elo points improvement is becoming rare for me. Most of >>the changes I try make the program weaker, and many changes do not provide any >>measurable improvement! >> >>That's why not having a strong test methodology is totally out of question if >>you are serious about chess programming. > >Devoting a whole week with 5 computers working 24/7 is a luxury few can afford. >During the past two years I have developed Falcon from a 2200 engine to a 2700+ >engine it currently is, all on one humble P3 733MHZ machine. > >In order to reach a 2700 level, the search should already be good enough. But >beyond that level, it is mostly the evaluation that matters. Since the Graz >WCCC, I have been spending almost all my time working on evaluation function. >The work on search has been limited to modifying one pruning here, one extension >there, etc. But again, beyond 2700, it is evaluation that matters. And I fully >agree with Vincent on that. > >It is almost impossible to test a single evaluation change to see whether it >improved the strength. If you change the evaluation of knight outposts by a few >centipawns, good luck testing it... In those cases you have to highly rely on >your feelings and chess knowledge, and then after doing many changes, test them >as a whole to see if they improved the strength. Just my two cents. > > Omid, I'm curious, how many NPS does Falcon do? (Of course give hardware.) I take it from the above that your search is essentially null-move based (possibly except near the leaves). I have the theory that there are three viable approaches to making a top engine: 1) ultra-high NPS, brute force (ie null move, some stuff at & below tips) 2) ultra-high NPS, selective 3) moderate NPS, ultra-selective Somehow, moderate NPS brute-force doesn't make much sense to me. Of course, practice should drive theory so there might be room in the above for a #4. :-) Vas > >> >>Even with a good test methodology chess programming is still an art: in many >>cases you have to decide with your feelings, because the raw data does not give >>you a definite answer. >> >>Now of course there are small improvements that I do not even need to test for a >>long time: if I find a way to make my program 10% faster without changing the >>shape of the tree, then all I need to do is run some safety tests that will only >>look at the number of nodes searched on a large set of positions and compare it >>to the last stable version. >> >>But that does not happen often! :) >> >> >> >> Christophe
This page took 0.02 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.