Author: Vincent Diepeveen
Date: 07:21:12 09/13/01
Go up one level in this thread
On September 12, 2001 at 19:28:15, Dann Corbit wrote: >On September 12, 2001 at 19:13:13, Bruce Moreland wrote: > >>On September 12, 2001 at 18:38:07, Slater Wold wrote: >> >>>I tested Crafty against the 300 WAC positions, yesterday and today. >>> >>>The tests were run with 48M of hash, and 10M of hashp. Also all 5 piece TB were >>>in place, and the machine was a 2x1.4Ghz that gets roughly 1.41M nps on the >>>bench test. (That's with the hash. With nothing but smpmt 2, it gets 1.44M.) >>> >>>1 second = 275/300 (Interestly, Deep Shredder get 274/300 in this test.) >>> >>>3 seconds = 293/300 >>> >>>5 seconds = 294/300 >>> >>>10 seconds = 295/300 >>> >>>60 seconds = 296/300 >> >>WAC is a dumb test when you have a program at this level. You can throw out all >>but about 10 of them, and you still have something that's hard to use to test a >>program at a sensible time control, because the remainder are either too easy >>(still) or too hard. > >I think you're wrong. >In fact, I think that this test may have uncovered some problem with crafty. >I used to get a better score on a slower machine. I suspect some minor thing >may have got broken. Either than, or an evaluation term may have been changed >that plays better in real games but plays worse on test suites. In that case it's a waste of time to use the WAC positions to test that. There are more effective methods than WAC to show uncovered problems! >I don't think you can throw out any of them, unless it gets them right. How >will you know it got them right unless you run it. >As a side point, sometimes chess engines discover new solutions to the problems >when run at longer time controls. That is an interesting finding, to me at >least. I find it nonsense, of course in completely won positions any move will win. Even nullmoving and a move later playing a good one is enough to win. If you are busy with WACs you keep wasting your time. get some stuff from informators then! >What are you trying to accomplish when you run a test suite?
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.