Author: Dann Corbit
Date: 22:13:34 12/06/01
Go up one level in this thread
On December 07, 2001 at 01:06:55, Russell Reagan wrote: >There are all kinds of statistics that you can formulate about a chess program, >the most notable being nodes per second. What statistic *should* imply a >stronger program? Obviously NPS is nice, but it's certainly not something you >can use to say that one program is stronger than another. I'm thinking that >number of plys would be a better indicator of strength, and I toyed with the >idea of some kind of nodes/ply kind of statistic. > >The nodes/ply statistic obviously would give different results in different >positions, but for testing your own program, it seems like a good tool. For >example, if you did an 8 ply search and searched N nodes without the use of a >transposition table, and after implementing the transposition table you only >searched N/2 nodes (I have no idea what the actual gain would be) then that's >obviously good. Then again, that indirectly implies a deeper search, which makes >this statistic boil down to plys/sec (in the same position of course, with the >same program, otherwise results would vary widely). > >Any ideas about other statistics or comments about my thoughts I've thrown out >here? There are lots and lots of statistics you can use. All of them are pretty well useless except won/loss/draw against opponents of known strength. Here are some reasons why: Suppose that I have a material only evaluation, and absurd pruning rules. I can search like the burning blue blazes, but it will play like total crap. Suppose that I tune my engine to run EPD test suites. I may get better performance on those suites, but far worse in actual games. Suppose that I have a slow searcher. The NPS comparisions make no sense against a fast searcher. Plys are probably one of the best indicators. However, if the eval is no good, neither are the plys worth anything. Test suites are a fairly good indicator -- but they are a necessary and not sufficient condition for powerful game play strength. In short "the proof of the pudding is in the eating."
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.