Author: Tord Romstad
Date: 17:45:08 05/25/04
Go up one level in this thread
On May 25, 2004 at 20:09:57, Andrew Wagner wrote: >On May 25, 2004 at 19:36:04, Will Singleton wrote: >> >>I'm surprised Bob would say that profiling is important so soon in the >>development process; perhaps there's some missing context. Profiling is, imho, >>about the last thing you'd want to do. I completely agree. I have occasionally profiled my program, mostly out of curiosity, but I am still nowhere near the stage where it is time to work on low-level optimization. There are still tons of bugs to fix, and countless possible ways to improve the search. Writing a fast program is not mainly about increasing the N/s count by a few percent, it is about searching the smallest possible tree without introducing too many errors. >Here's the link, so you can read it in context, if you'd like: >http://chessprogramming.org/cccsearch/ccc.php?art_id=112972 > >> >>1. Fix bugs in movegen, using perft tests. > >He does say "once it plays legally". To me, that implies a bugfree movegen... > >>2. Write a very simple, bug-free eval. >>3. Concentrate on move-ordering, which is crucial to making the tree small. >>Develop methods for measuring the quality of your ordering, don't only look at >>node counts. >> > >I would count these as what he calls "hot spots". Especially move ordering >(though good eval helps move ordering). Perhaps, but they are not the kind of hot spots you would find by using a profiler. Because Bob mentioned the technique of comparing the number of nodes searched before and after some optimization is implemented and making sure that the two numbers are the same, I doubt that he was talking about things like move ordering. >>Don't spend a lot of time on arcane or new ideas until you're certain what you >>have is bug-free. Especially make sure your transposition code is simple and >>effective, tons of problems result from bad hashing. > >I would also be interested in a process for this. What process do you use to >really be absolutely sure your program is bug free? Especially your hash table >code. E.g. at the moment, I think Trueno is bug free, more or less. But I >haven't found a good thorough test to give it that will tell me. Unfortunately I don't know to make sure my program is bug free. I have plenty of bugs, even a couple of simple, obvious, eye-catching and very serious-looking bugs which for some mysterious reason appear to make the program much stronger (every time I try to make the one-line change needed to fix the bugs the playing strength drops by about 100 Elo points). But here are a couple of techniques I have found useful: 1. Whenever you try out some new pruning technique (for instance when you introduce hash table pruning), don't add the actual pruning to the engine immidiately. Instead, set a flag which says "I would have pruned this position if my new trick was enabled" (for instance, if the hash table entry for the position contains a lower bound which is higher than beta, and the depth of the entry is sufficiently big). If it later turns out that the pruning decision was wrong (in the above example, if the search doesn't fail high, despite the lower bound >= beta in the hash table), you probably have a bug. Write the position to a log file. By inspecting the log file, you should be able to spot the most serious bugs. This technique is not only useful when adding non-speculative pruning, but also when implementing various sorts of speculative forward pruning. It can be used not only to find bugs, but also to detect holes and missing special cases in forward pruning tricks. 2. Write a function to compute a mirror image of the position on the board (i.e. black and white sides interchanged). Find a big file of EPD positions, and let your engine parse the file and evaluate each position and its mirror image. This helps you find asymmetry bugs in the evaluation function. This test should be done almost every time you do a change in the eval. It is surprising how easy it is to introduce new asymmetry bugs. Tord
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.