Author: Bruce Moreland
Date: 14:36:25 01/18/98
Go up one level in this thread
On January 18, 1998 at 15:59:01, Don Dailey wrote: >The new ccc test suite looks like it might be reasonable after we >clean it up and take out the easy ones. I looked at my counts >for 1, 2, 4, 8 and 16 seconds and get a nice improvement curve >so it looks like the difficulty is reasonably well distributed >which is a good characteristic for a problem set in my opinion. > >At 1 second(s) Cilkchess solves 349 >At 2 second(s) Cilkchess solves 385 (35) >At 4 second(s) Cilkchess solves 433 (48) >At 8 second(s) Cilkchess solves 481 (48) >At 16 second(s) Cilkchess solves 525 (44) > >The number in parenthesis is how many extra problems were picked up. > >Bob says Crafty gets 569 at 20 seconds. Cilkchess gets 539 at >20 seconds which is a little discouraging considering at least >a 3 to 1 hardware advantage. But Cilkchess is a very slow "evaluation >bound" program and is extremely conservative about extensions so >maybe this is not too horrible. > >I will now start looking for cooks and multiple solutions. I have stayed out of this, not because I don't care, but because I am doing a lot of research right now and I have three computers busy 24 hours per day. If I am saying something that is redundant or wrong, please feel free to ignore me. ECM is 879 positions, which is inconvenient to run in more than perhaps 20 seconds. If you guys can trim out positions that you all agree are easy, the test suite will be cut by a large percentage, and longer runs can be done, perhaps to cut it again. If you guys produce a list that is 1/4 to 1/3 the size we have now, it should be possible to examine individual problems in greater detail. You can produce a shorter list by deleting problems that you all get quickly, then by running these for a a little longer, and deleting all of the remainder that you all get in a little more time. You can do this without actually thinking about any of the positions that you are cutting. If you all get a problem pretty quickly, it is either easily solvable or something is wrong with it. If someone misses it, just leave it alone until you can all run it for longer. With a smaller set you or we or whoever can run for a longer period of time per problem, and try to get an idea which of the remaining problems are really broken. It's better to say a problem is cooked or broken after you've run it for five minutes than after you've run it for 20 seconds, right? bruce PS. I have ECM runs for about 60 versions of Ferret (20 seconds each). Would you like me to give you a list of which problems are solved by *all* versions after N seconds?
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.