Author: Dann Corbit
Date: 12:29:11 09/13/01
Go up one level in this thread
On September 13, 2001 at 15:14:11, Bruce Moreland wrote: >On September 13, 2001 at 13:29:00, Dann Corbit wrote: > >>The approach used in Beowulf is to calculate average time to solution and other >>statistical data of that nature. That way, if you can reduce solution time, you >>can still glean data even from the easiest possible data set. > >Yes, but in large part you are dealing with solution times that are measured in >fractions of a second. > >It might be valid to look at this kind of thing. If you run these for one >second each, perhaps you get some information about how your tree works near the >tips. > >This isn't the kind of suite you'd use to see how you'd behave at tournament >time controls. > >And you can't use it to evaluate changes to SMP search, since that gets much >more efficient if you leave it running for a while. > >If you are talking about letting solved positions run for a while, in order to >get information from that, these aren't the positions I'd choose, since I'd >rather optimize time taken to find the tactical move, or time taken to get to a >deep depth in a positional case. After the game is over, it doesn't matter how >deep you get. Good point. Of course, the general technique works for *any* test suite, since it is built into the program. I suppose I am fascinated with WAC for several reasons; among them: 1. It is a yardstick with good correlation between score and program ability IOW, programs that score less than 280 are not top programs. Programs that score less than 250 are easily beaten. 2. It is the only test suite that has been thoroughly debugged. I think it likely that some of the more sloppy test suites could give false results. You scored better, but actually, there is a bug in your program and it should have done worse. I don't think enough has been done to produce large, tough, debugged test suites. I suppose at some point, the harder test suites will be a lot more interesting to me (like when I can begin to solve them) ;-)
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.