Author: Vincent Diepeveen
Date: 10:23:08 09/04/02
Go up one level in this thread
On September 04, 2002 at 13:10:25, Robert Hyatt wrote: No one uses these speedup numbers except you bob. Not clearing the hashtable is not a fair way to test, but of course a great way to get a better speedup than you actually get. Also we watn the node counts obviously and as crafty is one of the few with a asymmetric king safety it's also better to turn that off by using the 'analyze' feature. >On September 04, 2002 at 12:25:09, Robert Hyatt wrote: > >>On September 04, 2002 at 12:11:53, Slater Wold wrote: >> >>>Bob, do you have those 20 positions from the ICCA article? >>> >>>If so could you e-mail or post them here for me? I would greatly apprecaite it. >> >> >>you mean the 24 positions from the MchessPro vs Cray Blitz game that I used >>in the JICCA article? If so, here they are... >> >>r2qkbnr/ppp2p1p/2n5/3P4/2BP1pb1/2N2p2/PPPQ2PP/R1B2RK1 b kq - ; bm a1a1 >>r2qkbnr/ppp2p1p/8/nB1P4/3P1pb1/2N2p2/PPPQ2PP/R1B2RK1 b kq - ; bm a1a1 >>a1a1 r2qkbnr/pp3p1p/2p5/nB1P4/3P1Qb1/2N2p2/PPP3PP/R1B2RK1 b kq - ; bm a1a1 >>black a1a1 >>r2qkb1r/pp3p1p/2p2n2/nB1P4/3P1Qb1/2N2p2/PPP3PP/R1B1R1K1 b kq - ; bm a1a1 >>a1a1 r2q1b1r/pp1k1p1p/2P2n2/nB6/3P1Qb1/2N2p2/PPP3PP/R1B1R1K1 b - - ; bm a1a1 >>black a1a1 >>r2q1b1r/p2k1p1p/2p2n2/nB6/3PNQb1/5p2/PPP3PP/R1B1R1K1 b - - ; bm a1a1 >>r2q1b1r/p2k1p1p/2p5/nB6/3Pn1Q1/5p2/PPP3PP/R1B1R1K1 b - - ; bm a1a1 >>r2q1b1r/p1k2p1p/2p5/nB6/3PR1Q1/5p2/PPP3PP/R1B3K1 b - - ; bm a1a1 >>r2q1b1r/p1k2p1p/8/np6/3PR3/5Q2/PPP3PP/R1B3K1 b - - ; bm a1a1 >>r4b1r/p1kq1p1p/8/np6/3P1R2/5Q2/PPP3PP/R1B3K1 b - - ; bm a1a1 >>r6r/p1kqbR1p/8/np6/3P4/5Q2/PPP3PP/R1B3K1 b - - ; bm a1a1 >>5r1r/p1kqbR1p/8/np6/3P1B2/5Q2/PPP3PP/R5K1 b - - ; bm a1a1 >>5r1r/p2qbR1p/1k6/np2B3/3P4/5Q2/PPP3PP/R5K1 b - - ; bm a1a1 >>5rr1/p2qbR1p/1k6/np2B3/3P4/2P2Q2/PP4PP/R5K1 b - - ; bm a1a1 >>5rr1/p2qbR1p/1kn5/1p2B3/3P4/2P2Q2/PP4PP/4R1K1 b - - ; bm a1a1 >>4qRr1/p3b2p/1kn5/1p2B3/3P4/2P2Q2/PP4PP/4R1K1 b - - ; bm a1a1 >>5qr1/p3b2p/1kn5/1p1QB3/3P4/2P5/PP4PP/4R1K1 b - - ; bm a1a1 >>5q2/p3b2p/1kn5/1p1QB1r1/P2P4/2P5/1P4PP/4R1K1 b - - ; bm a1a1 >>5q2/p3b2p/1kn5/3QB1r1/p1PP4/8/1P4PP/4R1K1 b - - ; bm a1a1 >>5q2/p3b2p/1k6/3QR1r1/p1PP4/8/1P4PP/6K1 b - - ; bm a1a1 >>5q2/p3b2p/1k6/4Q3/p1PP4/8/1P4PP/6K1 b - - ; bm a1a1 >>3q4/p3b2p/1k6/2P1Q3/p2P4/8/1P4PP/6K1 b - - ; bm a1a1 >>3q4/p3b2p/8/1kP5/p2P4/8/1P2Q1PP/6K1 b - - ; bm a1a1 >>3q4/p3b2p/8/2P5/pk1P4/3Q4/1P4PP/6K1 b - - ; bm a1a1 >> >>Note that these were produced by Vincent. I had to edit them as he had >>bad castling status. Note also that the "best move" is nonsense, but in >>these positions, "best move" is not important. >> >>How to run the test: >> >>1. set some search time limit with the st=n command. >> >>2. type "test filename" where the above file is used. >> >>Run this with mt=0 and mt=2. >> >>Now comes the fun. For each position in the mt=0 run, find the _last_ >>bit of output, such as a PV change or the end of iteration. Write the >>time down. Flip to the mt=2 run and for that position, find the _same_ >>depth/score/move output and write down that time. Now you can compute >>a speedup for that position. >> >>Repeat for all 24 positions and you have 24 speedups. >> >>Now you come to a decision point. If you do as I said above, you will then >>average the 24 speedups to get an overall speedup. But this is just one way >>to do it, because this counts all positions equally in the final speedup. >>But suppose some positions have a much longer "time" value? In a real game, >>won't their speedup affect the overall average "more"? So you might choose to >>go back and take the times as derived above, add them together, and _then_ >>divide the mt=2 time into the mt=0 time to compute an overall speedup. >> >>There is also another way but it is not as easy... >> >>use st=999999, sd=N to specify a specific search depth for every position to >>be searched to. Then you can use the total elapsed time for mt=0 and divide >>that by the total elapsed time for mt=2, and compute the speedup that way. >>The drawback is that if you pick too deep a search, some positions will take a >>_long_ time. If you pick it too shallow, some will finish very quickly. >> >>You _can_ get at least three different speedup values from the above, and which >>one you use is really optional... > > >BTW, if you run this, please post the results. One note is that these positions >are consecutive positions from a game. Which means that Crafty will not be >clearing the hash table between each position, just like it doesn't in a real >game. > >The only downside here is that when playing a real game, if Crafty completes a >12 ply search, then it will start pondering at 11 plies. Because when you >remove the move played in the game, and the pondered move, you are left with >10 plies of good search and starting at 11 is the right thing to do. Testing >using the "test" command doesn't mimic this behavior, the search will always >start at ply=1 and go forward. many times it will find good stuff in the hash >table and get back to depth=11 almost instantly. Other times it won't, and the >results will be a little better (or worse) than reality in those cases. > >There is so much variability in this it isn't funny...
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.