Author: Bruce Moreland
Date: 09:22:38 06/09/98
Go up one level in this thread
On June 09, 1998 at 11:57:36, Bruce Moreland wrote: >90% is 270, 95% is 285. >85% is 255, 98% is 294. I thought some more about this. This is actually a pretty good example of when it is a good idea to use a test suite. Thorsten, in some other post, makes an interesting point, which I will expand upon a bit. These tactical test suites will tell you how good a program is at reaping a tactical reward, but they won't tell you how good a program is at creating a situation where a tactic exists, and for that matter they won't tell you how good it is a avoiding a situation where a tactic exist for the other side. So if you want to eat and drink test suites, there are some obvious deficiencies. If your program is not aggressive, it doesn't get to show off its tactical ability. Another thing test suites don't show you is how well a program is able to play the whole game. It doesn't do much good if you achieve a won position, and you find the winning sacrifice, or whatever, if you are unable to convert the material-up ending that follows. But test suites are still useful. If you run one of the Elo suites and try to tune for a few pseudo-Elo points, I think this is not much use. But you can use them to try to identify inefficiencies in the search. If your goal when you make a change to your program is to improve general tactical speed, you probably didn't do it if your solution curve on a normal tactical test suite is significantly worse. Your results will be wrong if what you really did was change the program's overall personality, but if all you did was try to change the shape of the tree a little bit, I think you can get a good idea of whether you did a good job or not, just by running one of the larger suites. bruce
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.