Author: Don Dailey
Date: 13:17:35 12/09/97
Go up one level in this thread
On December 08, 1997 at 09:12:24, Robert Hyatt wrote: >On December 08, 1997 at 01:27:12, Jouni Uski wrote: > >>I have Kurt Richter's book "Kombinationen" 7. Edition. It contains about >>400 tactical positions. Most are too easy for today programs. But there >>is >>14 positions, which cause problems at least to some: >> >> >>2k4r/pp3ppp/2p2n2/5PBb/P3P3/2Nr3P/1PB2bP1/R3R2K b - - id "10"; >>rn2kb1r/p3qppp/2p2n2/1p2p1B1/2B1P3/1QN5/PPP2PPP/R3K2R w KQkq - id "21"; >>r3rnk1/pbq1b1pp/1pp1pp2/4N3/2PP2Q1/3B4/PP1B1PPP/R3R1K1 w - - id "71"; >>3r2rk/2q2p1p/p2p1pn1/1p1PnN1Q/2p1P3/2P3B1/PPBN2K1/R7 w - - id "77"; >>r4rk1/1R3p1p/p3pBpP/qb1pP3/2nP4/P2Q1N2/5PP1/2R3K1 w - - id "126"; >>q5rr/1b2b1k1/pp2p1p1/2p1PpP1/2P2P1R/2BB3R/PP5Q/4K3 w - - id "146"; >>r6r/p2Rbk1p/2p2p2/q6b/8/2N2N1P/P1PP1PP1/2BQ2K1 w - - id "205"; >>2r2rk1/1p1bbppp/p2ppn2/q7/4PP2/2N1BB2/PPP3PP/R3QR1K w - - id "224"; >>rn1q1r2/pb3pk1/4p1p1/4P1P1/1ppP1N2/8/PPB2PP1/R2QK2R w KQ - id "257"; >>r4k1r/ppp3pp/2n1bp2/7B/q3PB2/2Q5/PP1R1PPP/2KR4 w - - id "266"; >>r1bn1rk1/5p1p/1qp2p1Q/ppb1pP2/3pP2N/P1NP2P1/BPP1K2P/R4R2 w - - id "297"; >>r1b1r1k1/1p1n1p1p/2ppnqp1/8/p1P1P3/5P2/PbNQNBPP/1R2RBK1 w - - id "328"; >>r1bqnrk1/pp1pppbp/6p1/n3P3/3N4/1BN1B3/PPP2PPP/R2QK2R w KQ - id "364"; >>1k6/p7/1p3P2/3QP3/8/4q3/6PP/2r2R1K w - - id "398"; >> >>Solutions: >> 1 Rxh3+ >> 2 Nxb5 >> 3 Bxh7+ >> 4 Qxh7+ >> 5 Qxg6+ >> 6 Bxf5 >> 7 Nd5 >> 8 Nd5 >> 9 Rh7+ >>10 Rd7 >>11 Na4 >>12 Rxb2 >>13 Bxf7+ >>14 Qf3 >> >> >> Fritz 4.01 Rebel 9 Mchess 6 >> >> 1 0:40 0:18 0:05 >> 2 6:47 -- 4:50 >> 3 -- 6:18 5:47 >> 4 0:18 0:10 0:04 >> 5 -- 2:04 0:16 >> 6 0:01 0:45 0:18 >> 7 -- -- -- >> 8 2:47 1:14 3:46 >> 9 0:52 3:19 0:34 >>10 1:20 1:29 1:10 >>11 5:31 -- 8:32 >>12 3:23 -- 0:08 >>13 0:01 1:41 0:26 >>14 1:48 9:06 0:47 >> >>I have tested in 486/150 Mhz PC. Times are to find AND hold. >>For some reason Mchess is best even if it speed is only about 7000 NPS. >>I am interested about Crafty, Ferrer, Nimzo and Shredder solution times. >> >>Jouni > >Crafty 14.3: > >1 Mat06 3.97 seconds >2 +1.00 .91 seconds >3 not found in 3 minutes >4 Mat07 1.55 seconds >5 +0.00 1.04 seconds >6 +3.60 3.24 seconds >7 +2.70 (Qe1) 4 seconds) (incorrect) >8 equal .15 seconds >9 +5.20 52.33 seconds >10 +1.32 2:19 >11 +1.00 (Nd1) .01 seconds. (incorrect) >12 not solved >13 0.00 .31 seconds >14 +5.60 9.66 seconds > >The evals are not the best score seen, rather they are the >score at the time crafty changed to the "solution" move. IE >it is possible/probable that at deeper searches, the score got >even better, but this gives an idea of what it saw at the time >it decided the solution move was best. Note that the time was >not recorded until it "held" the correct move from that time to >the end of the search (I gave a 4 min max)... Cilkchess serial version on Alpha 466 Mhz # KEY TIME --- -------- ---------- ------------------------------ 1) Rxh3+ 11.4 Mate in 6 2) Nxb5 6.0 Solved positionally? 3) Bxh7+ 2872.7 4) Qxh7+ 9.3 Mate in 7 5) Qxg6+ 32.9 6) Bxf5 1.3 7) Nd5 -- We get a score jump with Qe1 in 1.3 seconds 8) Nd5 0.0 instant - score jumps 1/3 pawn or so 9) Rh7+ 145.7 10) Rd7 60.2 11) Na4 -- Plays Nd1 with score jump in 6 seconds 12) Rxb2 170.8 13) Bxf7+ 0.5 1/2 pawn jump for this move 14) Qf3 -- No solution These are fun to run but I don't think problem solution times tell you enough about the strength of a chess program. We discovered long ago that it is easy to make your program perform well on problem sets but most of these "enhancements" weaken the program too much. A few simple well known algorithms might make 3 or 4 of these problems solve an order of magnitude faster and would imply a huge improvement to the strength of the program when it truth the program has been slightly weakened. Why is this so? I have a theory about this. Essentially, when you give some move extension you are dramatically improving the programs performance in a very limited subset of actual positions you will encounter in real chess play. BUT AT THE SAME TIME, you are weakening the program slightly in ALL positions where the extension does not provide additional insight and this is MOST positions. So you are playing most of the game slightly weakened, hoping for a won position to show off your tactics (you must have a won position for any tactics to be successful.) I've talked to a lot of programmers about this and a lot of them don't seem to mind the 30 or 40 percent slowdown they get, they pass it off as being completely unimportant compared to finding checkmates or other flashy tactics. But I think it's critically important. Every move the program makes involves some battle for something, and that something might be just a minor positional thing, such as getting to castle a little earlier, or getting an extra square of mobility for the bishop etc. It does not take much of a slowdown before a chess program starts losing a significant number of these "little skirmishes." Most observers, including the programmers themselves will notice that their beloved program lost the war but never notice how many battles were lost. We simply rarely notice the little tactics of positional play, and if we do, we tend to believe that the programs' evaluation function itself was in error. Also keep in mind that most problem sets begin with unatural positions, namely positions where one side already has a win and the game is essentially over if the program can find the "right" move. There do seem to be some extensions that work pretty well that have been discovered. They are generally the type of moves that affect a really large percentage of positions (such as recaptures and checks.) There are some other techniques that can be applied in the quies search that can pick up problems several ply earlier in some cases which I'm pretty sure Mchess leans heavily upon. But it's unclear to me if they really are a benefit. It seems clear that most programs differ quite significantly and they all choose different sets of tradeoffs. That's what makes it so much fun! I once noticed that the top rated programs on the rating list varied significantly in their tactical abilities. In some cases the best program was significantly worse (relative to the top few) in problem solving ability! Occasionally we've also seen exceptional problem solvers with mediocre ratings. There is a definite correlation between problem solving ability and chess strength, but it does not seem to be as strong as our intuition suggest it should be. -- Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.