Author: Don Dailey
Date: 08:35:04 12/30/97
Go up one level in this thread
On December 30, 1997 at 07:38:54, Lucien Okkes wrote: >Hello everyone, > >Some time ago I started to write my own chess program. At this moment >the program seems to work fine, but I am not convinced that it is >bug-free. Has anyone suggestions for thoroughly testing a chess program? >I'm not the first one doing this, so I think there should be some kind >of existing testprocedures!! > >Lucien Do you mean testing for strength or for bugs? Either way you have a tough job ahead but it is great fun. Here are a few things I do and recommend for bug catching/testing: 1) Use lots of assertions. I don't use these as much as I should but am often rewarded when I do. 2) Write routines twice. For instance I have a slow check test routine that I implemented the simple way with no tricks. I use it to test my fast routine. 3) Use the "parity" test. Put a routine in your program that switches the board around so that white is black and black is white. Does your program score the positions the same? You should almost always get the same move back too but this is not neccesarily guaranteed. If you don't, then look for a bug. A version of this is to play sequences like e3 e5 e4 to get an "e4 e5" position with colors reversed and test this for consistancy. 4) Have a deterministic mode in your program. In my deterministic mode I clear hash tables before every move, as well as killer and history information and anything that will prevent it from being deterministic. So when I implement many algorithm (like fast check testing again) the program better look at exactly the same number of nodes or something is wrong. 5) Try to write code that is "unforgiving" in the sense that if something goes wrong, the program will crash etc. This is similar to using lots of assertions (I am ussuming you program in C.) 6) Run problem sets you are very familiar with. You should know exactly how many ply your program should take for each problem taking into consideration all extensions and other search and scoring behavior. I catch many bugs this way. 7) Run lots of auto/self testing where the program simply plays lots of games with itself. Early versions of Cilkchess would crash once every 200 games or so until I fixed some subtle bugs that rarely showed up. Spot check the games too, I once played a long sequence of games without any crashes but when I looked at the games I saw it making silly tactical blunders but only occasionally. This led me to another bug find. 8) Compare. Keep stable versions around and self test against them. When the results are not very close to even after long test sequences you have either a bug or you introduced a weak playing algorithm. 9) Put in lots of switches. Whenever I detect a problem it is often useful to turn off hash tables, selectivity, quies tricks, etc and find out who the culprit is. This is quite useful sometimes. I hope this helps. I also have lots of debugging routines in my program sitting there and waiting to be used. One routine will throw up an xboard on the screen with a position you pass to it. It will also display lots of useful stuff to help me debug. -- Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.