Author: Russell Reagan
Date: 21:14:25 05/25/04
Go up one level in this thread
On May 25, 2004 at 17:44:55, Andrew Wagner wrote: >I do a lot of reading through CCC archives. I use the search engine from here, >and also I'm in the process of reading through the old archives systematically >using the offline reader (I'm in the fall of 2001 currently, I think). Anyway, >sometimes I run across a nugget that makes me just stop and go "whoah". Here's a >quote from one of Bob's posts, originally about hashing algorithms: > >>I think the key to improving a program, once it plays legally, is to develop >>a methodology to carefully profile the code, find the hot spots, and then find >>ways to speed up those hot spots. But all the while paying _careful_ attention >>to the overall node counts on a wide range of test positions. A 1% speedup is >>of no use at all if you introduce an error that happens once every billion >>nodes. I can search that many nodes in 15 minutes. I can't stand errors that >>frequently. I have what would probably be called a "zero-tolerance for errors" >>in Crafty. If I make a change that should only make it faster or slower, then >>the node counts must remain constant. If they don't I debug until I find out >why and fix it. > >This is a fantastic point. Maybe somewhat obvious to our more experienced >members, but certainly words of wisdom for us newbies. So, my question is, what >methods are you all using for profiling your code? How do you go about >identifying and fixing your hotspots? Do you have a particular test suite you >use, or what? Andrew One minor point of clarification. I think a better subject line would have been: "The key to improving a program's speed". There are many aspects of a chess program that we can improve. How fast it runs, what knowledge it has, which variations to search, and on and on. This quote mainly deals with improving the "how fast" part. Profiling, detecting hot spots, and eliminating hot spots will improve your program to some degree, but it is not the key to improving a program's playing strength. You can spend the rest of your life optimizing TSCP, but it will never be a top engine if you only focus on improving the speed, and at some point you won't be able to improve the playing ability by improving the speed of the program. I'm sure most readers assumed this, but I had to point it out just in case. I wasted years worrying about the best way to improve the "how fast" part of a program. Then one day I learned that the "how fast" isn't the most important thing. Actually, it's closer to the end of the list. However, the key to improving a chess program is similar to the "profile, find hot spots, eliminate hot spots," approach to improving speed. The basic idea for improving anything is that to know that you have improved it, you have to be able to measure the improvement. To know you have made the program faster, you have to be able to measure that. To know you have made the program play better, you have to be able to measure that also. I don't remember if it was Christophe Theron or Ed Schröder (or someone else), but someone with the skins on the wall to back up their claim said that the real secret of the commercial authors, and of how to improve the program, is to have a reliable way to test whether a change you make to your program is an improvement or not. You can make plenty of change that you *think* will result in an improvement, but I can say from experience that a lot of "fantastic" ideas I've had turned out to be complete duds after testing them. If you can scientifically measure that a change is an improvement, then you have a reliable way of moving forward and making progress. Otherwise, you might be taking a few steps forward, or a few steps back, and you won't have any idea which.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.