Author: Don Dailey
Date: 15:06:49 01/18/98
Go up one level in this thread
Hi Bruce, I think we are doing the right thing. I agree we should cull the set right away and kill the ones that we are all finding easy. That will make it a lot easier to work with the remaining ones. We are gradually analyzing the cooks and multiple solutions until we are convinced we have good problems but we should probably wait for the set to shrink first. I sent a list of all my "less than 10 second" problems to Bob via email. I sent it in the orignial epd format hoping this would make it easier for him to cull (using unix tools I assume like uniq etc.) I discovered that if I get rid of the "simple" ones and the "flaky" ones (unsolves at some point) I am already left with only 302! You should probably email your results to Bob too in case he wants to use them too. - Don On January 18, 1998 at 17:36:25, Bruce Moreland wrote: > >On January 18, 1998 at 15:59:01, Don Dailey wrote: > >>The new ccc test suite looks like it might be reasonable after we >>clean it up and take out the easy ones. I looked at my counts >>for 1, 2, 4, 8 and 16 seconds and get a nice improvement curve >>so it looks like the difficulty is reasonably well distributed >>which is a good characteristic for a problem set in my opinion. >> >>At 1 second(s) Cilkchess solves 349 >>At 2 second(s) Cilkchess solves 385 (35) >>At 4 second(s) Cilkchess solves 433 (48) >>At 8 second(s) Cilkchess solves 481 (48) >>At 16 second(s) Cilkchess solves 525 (44) >> >>The number in parenthesis is how many extra problems were picked up. >> >>Bob says Crafty gets 569 at 20 seconds. Cilkchess gets 539 at >>20 seconds which is a little discouraging considering at least >>a 3 to 1 hardware advantage. But Cilkchess is a very slow "evaluation >>bound" program and is extremely conservative about extensions so >>maybe this is not too horrible. >> >>I will now start looking for cooks and multiple solutions. > >I have stayed out of this, not because I don't care, but because I am >doing a lot of research right now and I have three computers busy 24 >hours per day. If I am saying something that is redundant or wrong, >please feel free to ignore me. > >ECM is 879 positions, which is inconvenient to run in more than perhaps >20 seconds. If you guys can trim out positions that you all agree are >easy, the test suite will be cut by a large percentage, and longer runs >can be done, perhaps to cut it again. > >If you guys produce a list that is 1/4 to 1/3 the size we have now, it >should be possible to examine individual problems in greater detail. > >You can produce a shorter list by deleting problems that you all get >quickly, then by running these for a a little longer, and deleting all >of the remainder that you all get in a little more time. > >You can do this without actually thinking about any of the positions >that you are cutting. If you all get a problem pretty quickly, it is >either easily solvable or something is wrong with it. If someone misses >it, just leave it alone until you can all run it for longer. > >With a smaller set you or we or whoever can run for a longer period of >time per problem, and try to get an idea which of the remaining problems >are really broken. > >It's better to say a problem is cooked or broken after you've run it for >five minutes than after you've run it for 20 seconds, right? > >bruce > >PS. I have ECM runs for about 60 versions of Ferret (20 seconds each). >Would you like me to give you a list of which problems are solved by >*all* versions after N seconds?
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.