Author: Dann Corbit
Date: 16:28:15 09/12/01
Go up one level in this thread
On September 12, 2001 at 19:13:13, Bruce Moreland wrote: >On September 12, 2001 at 18:38:07, Slater Wold wrote: > >>I tested Crafty against the 300 WAC positions, yesterday and today. >> >>The tests were run with 48M of hash, and 10M of hashp. Also all 5 piece TB were >>in place, and the machine was a 2x1.4Ghz that gets roughly 1.41M nps on the >>bench test. (That's with the hash. With nothing but smpmt 2, it gets 1.44M.) >> >>1 second = 275/300 (Interestly, Deep Shredder get 274/300 in this test.) >> >>3 seconds = 293/300 >> >>5 seconds = 294/300 >> >>10 seconds = 295/300 >> >>60 seconds = 296/300 > >WAC is a dumb test when you have a program at this level. You can throw out all >but about 10 of them, and you still have something that's hard to use to test a >program at a sensible time control, because the remainder are either too easy >(still) or too hard. I think you're wrong. In fact, I think that this test may have uncovered some problem with crafty. I used to get a better score on a slower machine. I suspect some minor thing may have got broken. Either than, or an evaluation term may have been changed that plays better in real games but plays worse on test suites. I don't think you can throw out any of them, unless it gets them right. How will you know it got them right unless you run it. As a side point, sometimes chess engines discover new solutions to the problems when run at longer time controls. That is an interesting finding, to me at least. What are you trying to accomplish when you run a test suite?
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.