Author: Tom Likens
Date: 19:12:04 05/25/04
Go up one level in this thread
On May 25, 2004 at 20:45:08, Tord Romstad wrote: >On May 25, 2004 at 20:09:57, Andrew Wagner wrote: > >>On May 25, 2004 at 19:36:04, Will Singleton wrote: >>> >>>I'm surprised Bob would say that profiling is important so soon in the >>>development process; perhaps there's some missing context. Profiling is, imho, >>>about the last thing you'd want to do. > >I completely agree. I have occasionally profiled my program, mostly out >of curiosity, but I am still nowhere near the stage where it is time to >work on low-level optimization. There are still tons of bugs to fix, >and countless possible ways to improve the search. Writing a fast program >is not mainly about increasing the N/s count by a few percent, it is about >searching the smallest possible tree without introducing too many errors. > >>Here's the link, so you can read it in context, if you'd like: >>http://chessprogramming.org/cccsearch/ccc.php?art_id=112972 >> >>> >>>1. Fix bugs in movegen, using perft tests. >> >>He does say "once it plays legally". To me, that implies a bugfree movegen... >> >>>2. Write a very simple, bug-free eval. >>>3. Concentrate on move-ordering, which is crucial to making the tree small. >>>Develop methods for measuring the quality of your ordering, don't only look at >>>node counts. >>> >> >>I would count these as what he calls "hot spots". Especially move ordering >>(though good eval helps move ordering). > >Perhaps, but they are not the kind of hot spots you would find by using a >profiler. Because Bob mentioned the technique of comparing the number of >nodes searched before and after some optimization is implemented and making >sure that the two numbers are the same, I doubt that he was talking about >things like move ordering. > >>>Don't spend a lot of time on arcane or new ideas until you're certain what you >>>have is bug-free. Especially make sure your transposition code is simple and >>>effective, tons of problems result from bad hashing. >> >>I would also be interested in a process for this. What process do you use to >>really be absolutely sure your program is bug free? Especially your hash table >>code. E.g. at the moment, I think Trueno is bug free, more or less. But I >>haven't found a good thorough test to give it that will tell me. > >Unfortunately I don't know to make sure my program is bug free. I have >plenty of bugs, even a couple of simple, obvious, eye-catching and very >serious-looking bugs which for some mysterious reason appear to make the >program much stronger (every time I try to make the one-line change needed >to fix the bugs the playing strength drops by about 100 Elo points). But >here are a couple of techniques I have found useful: > >1. Whenever you try out some new pruning technique (for instance when you > introduce hash table pruning), don't add the actual pruning to the engine > immidiately. Instead, set a flag which says "I would have pruned this > position if my new trick was enabled" (for instance, if the hash table > entry for the position contains a lower bound which is higher than beta, > and the depth of the entry is sufficiently big). If it later turns out > that the pruning decision was wrong (in the above example, if the search > doesn't fail high, despite the lower bound >= beta in the hash table), > you probably have a bug. Write the position to a log file. By inspecting > the log file, you should be able to spot the most serious bugs. This > technique is not only useful when adding non-speculative pruning, but > also when implementing various sorts of speculative forward pruning. It > can be used not only to find bugs, but also to detect holes and missing > special cases in forward pruning tricks. > >2. Write a function to compute a mirror image of the position on the board > (i.e. black and white sides interchanged). Find a big file of EPD > positions, and let your engine parse the file and evaluate each position > and its mirror image. This helps you find asymmetry bugs in the evaluation > function. This test should be done almost every time you do a change in > the eval. It is surprising how easy it is to introduce new asymmetry bugs. This is a great way to catch asymmetry bugs. Don't forget that you can mirror the board top-to-bottom *and* left-to-right. If you carry this out you will get three additional boards to compare against (four if you count the original position). 1. original position 2. T-2-B 3. L-2-R 4. T-2-B followed by L-2-R regards, --tom >Tord
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.