Author: Michel Langeveld
Date: 08:28:16 03/09/04
Last week I worked hard on Nullmover and the ecm-gcp testsuite in particulair. I thought this testsuite is a fast way to the tune middlegame knowledge, kick things out and get things back in and do multiple testruns. I did the following: *1 kicked my kingsafety totally and use only pawnshelter *2 kicked my mobility totally out *3 added more information in the pawn struct *1 gives me +- 10% performance back *2 gives me +- 10% performance back *3 gives me +- 5% extra performance. I know that nps is not important but on a P3-800Mhz most I get most of the time around 450-500knps. On a PIV 2.4Ghz machine I have most of the time 1000-1100knps. I tested: - testsuite: ecm-gcp.epd - timecontrol: 5 seconds - hardware: P3 800Mhz machine - hashtable: 50Mb P3 800: The old nullmover (v0.25) scored 58 positions right in 5 seconds. P3 800: The new nullmover (v0.25c) scores 75!! positions right in 5 seconds. I also tested on 10 seconds a move and there it is also an improvement: P3 800: The old Nullmover (v0.25) scores 78 positions right in 5 seconds. P3 800: The new Nullmover (v0.25c) scores 95!! positions right in 10 seconds. I made also another testing mechanism that emulates a game. It skips the opening and I use this to try the hashing algorithm: HASHTEST 10 Test01: 3080962 time= 6.31 FH= 96.6% hhit%: 6.1% a4xb6=27 Test02: 4408305 time= 8.92 FH= 94.4% hhit%: 7.1% f4xe5=23 Test03: 11364980 time=21.20 FH= 92.5% hhit%: 5.8% o-o=-4 Test04: 5378200 time= 9.73 FH= 93.4% hhit%: 6.3% h2-h3=25 Test05: 2149098 time= 4.02 FH= 95.3% hhit%: 7.2% d5-b3=26 Test06: 9823962 time=17.91 FH= 88.9% hhit%: 5.8% h2-h3=26 Test07: 2493285 time= 4.39 FH= 94.0% hhit%: 6.7% c1-g5=24 Test08: 2344532 time= 4.05 FH= 96.4% hhit%: 6.4% g5xf6=49 Test09: 5186665 time= 9.06 FH= 90.6% hhit%: 6.8% h2-h3=50 Test10: 2473415 time= 4.36 FH= 92.0% hhit%: 8.6% h2-h3=50 Test11: 1252083 time= 2.50 FH= 88.5% hhit%: 8.2% e2xf3=52 Test12: 4202884 time= 7.72 FH= 87.8% hhit%: 8.2% f3-h5=58 Test13: 4188662 time= 8.89 FH= 84.5% hhit%: 8.7% f3-h5=62 Test14: 2227369 time= 4.19 FH= 86.3% hhit%: 8.7% f2-c2=52 Test15: 3483631 time= 6.33 FH= 87.6% hhit%: 10.8% a2-a4=38 Test16: 2648911 time= 4.84 FH= 87.5% hhit%: 10.3% a2-a4=21 Test17: 2582746 time= 4.61 FH= 87.4% hhit%: 10.7% a2-a4=25 Test18: 1975938 time= 3.58 FH= 90.0% hhit%: 11.4% f2-e3=24 Test19: 2570348 time= 4.53 FH= 89.5% hhit%: 12.2% a2-a4=13 Test20: 2786429 time= 5.20 FH= 91.6% hhit%: 14.7% g3-e3=7 Test21: 1312456 time= 2.61 FH= 88.8% hhit%: 14.7% f2-h4=-1 Test22: 2294598 time= 4.26 FH= 87.2% hhit%: 13.5% a2-a4=4 Test23: 3625229 time= 6.95 FH= 87.3% hhit%: 9.7% e1-e2=5 Test24: 1322411 time= 2.72 FH= 88.5% hhit%: 10.5% e1-e3=-1 Test25: 543736 time= 1.33 FH= 93.3% hhit%: 9.0% e3-f3=-5 Test26: 2453827 time= 4.95 FH= 89.9% hhit%: 8.2% g1-h1=5 Test27: 2146519 time= 4.33 FH= 91.8% hhit%: 8.4% d5-a2=5 Test28: 1233153 time= 2.56 FH= 91.3% hhit%: 10.2% h2-f2=24 Test29: 560284 time= 1.25 FH= 89.7% hhit%: 13.1% g1xf2=22 Test30: 1673102 time= 3.27 FH= 85.9% hhit%: 11.1% d1-b1=37 Test31: 651264 time= 1.45 FH= 83.0% hhit%: 14.0% a2-d5=48 Test32: 491891 time= 1.16 FH= 84.7% hhit%: 13.3% g1-g2=51 Test33: 1812927 time= 3.36 FH= 87.0% hhit%: 11.8% g1-f2=53 Test34: 535576 time= 1.17 FH= 87.8% hhit%: 10.9% c3xd4=44 Test35: 783155 time= 1.83 FH= 91.2% hhit%: 15.0% e1-c1=70 Test36: 610921 time= 1.41 FH= 95.5% hhit%: 10.8% h3-h2=75 Test37: 618524 time= 1.38 FH= 92.3% hhit%: 10.6% h2-c2=80 Test38: 699858 time= 1.55 FH= 94.1% hhit%: 13.1% c1-d1=92 Test39: 488832 time= 1.11 FH= 96.7% hhit%: 9.5% c2xc3=324 Test40: 118758 time= 0.42 FH= 97.0% hhit%: 12.2% c1xc3=408 Test41: 180520 time= 0.56 FH= 94.9% hhit%: 14.7% c3-c7=409 Test42: 189391 time= 0.58 FH= 96.0% hhit%: 12.4% c7xf7=535 Test43: 102407 time= 0.41 FH= 93.1% hhit%: 13.1% f7xf6=591 Test44: 101539 time= 0.44 FH= 90.6% hhit%: 15.6% f6xd6=611 TOTAAL: 101173283 time=193.39 nps=523157 I expected a big elo improvement with this new version when playing comp vs comp games. The opposite happened. :-( The new version scores 122 elo!! worse. I tested this on the same hardware (P3 800Mhz) with a timecontrol 40 moves / 5 minutes games as the older version. 26 Nullmover 0.25 : 2019 58 50 148 50.3 % 2017 14.2 % 65 Nullmover 0.25c: 1897 115 85 44 30.7 % 2039 20.5 % I should play more games to have more more reliable elo rating but this is not what I expected. And it looks already so worse that futher testing doesn't make sense. I am very puzzled. Somewhere I should have made a mistake. Maybe my approach for tuning on ecmgcp.epd was wrong. My questions: * Is 5 seconds a move a good timecontrol to test ecmgcp.epd ? * Does it makes sense to use a reall long timecontrol? * I learnt that ecmgcp.epd is much more tactical as strategical. Does someone has other/ better positional/eval-tweaking advice? No testsuites? Other testsuite, like lct2.epd, the comeplete ecm.epd ?
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.