Author: Don Dailey
Date: 12:59:01 01/18/98
Go up one level in this thread
The new ccc test suite looks like it might be reasonable after we clean it up and take out the easy ones. I looked at my counts for 1, 2, 4, 8 and 16 seconds and get a nice improvement curve so it looks like the difficulty is reasonably well distributed which is a good characteristic for a problem set in my opinion. At 1 second(s) Cilkchess solves 349 At 2 second(s) Cilkchess solves 385 (35) At 4 second(s) Cilkchess solves 433 (48) At 8 second(s) Cilkchess solves 481 (48) At 16 second(s) Cilkchess solves 525 (44) The number in parenthesis is how many extra problems were picked up. Bob says Crafty gets 569 at 20 seconds. Cilkchess gets 539 at 20 seconds which is a little discouraging considering at least a 3 to 1 hardware advantage. But Cilkchess is a very slow "evaluation bound" program and is extremely conservative about extensions so maybe this is not too horrible. I will now start looking for cooks and multiple solutions. - Don On January 18, 1998 at 15:19:08, Don Dailey wrote: >Bob, Amir, others, > >I ran the full suite to approx 30 seconds. I assume Amir is >planning on running the set too to give us 3 programs. Is that >correct? We don't want too many running because we may end >up with too many easy problems since we want everyone to agree >on the easy ones. > >But we want everyone to find cooks, multiple solutions and >other difficulties that will allow us to throw out bad problems. >Bob Hyatt will be the final judge! Post your positions and >why you think it's bad so Bob can rule on it and we can all >see. > >I also recommend we partition the set so that we have a hard >group. ( ccc-hard.epd ) This could be any problems where >none of the programs solve in the 10 second period. > >When I ran the set I identified 3 flaky positions. I define >"flaky" as any position that is "solved" and unsolved once or >more (whether it's solved again or not.) I do not count >solutions on the 1st iteration because it is normal to see >bad moves before finding good ones. > >I haven't checked them out yet so maybe my program has a bug >or something but here they are for everyones inspection: > >3rr1k1/1p1qb2p/pP4p1/2p2p2/4RQ2/3P4/1PPB2PP/4R1K1 w - - bm Bc3; id >"ECM.420"; >6k1/rnqb3p/5ppQ/2pPp3/p1N1P3/2PB3P/5PP1/1R4K1 w - - bm f4; id "ECM.422"; >1q4k1/p3rppp/3b1n2/8/5P2/4P3/PP1B3P/RNQ3K1 b - - bm Bxf4; id "ECM.505"; > >Chances are if I ran deeper I would find more. Other programs >should check for this too, I'll bet each program finds different >ones. > >- Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.