Author: Uri Blass
Date: 12:29:46 07/17/02
Go up one level in this thread
On July 17, 2002 at 13:45:52, Dann Corbit wrote: >On July 17, 2002 at 07:11:20, Sune Fischer wrote: > >>On July 17, 2002 at 06:27:20, Dann Corbit wrote: >> >>>On July 17, 2002 at 05:56:52, Nicolas GUIBERT wrote: >>> >>>> >>>>>After reading the posts of others, I think that one thing should be made clear. >>>>>I think you should decide what kind of tournament this is going to be, then >>>>>develop the rest of it around it. For example, if it was going to be a "world >>>>>championship", then (IMO) the goal is to determine the best engine in the world. >>>>>If that is your goal, I would expect longer time controls... >>>> >>>>NO NO NO >>>> >>>>If you want to try and determine the best engine, do not play at longer time >>>>controls !!!!!! >>>> >>>>BUT... Do play a lot more games with shorter time controls. >>>> >>>>Then you will get a more reliable result ! >>>> >>>>I have never understood why computer chess competitions are so slow. For human >>>>players, this is understandable. For computers, it is just nonsense. I believe >>>>that this has to do with history. The first competitions had to be played slowly >>>>because the computers were terribly bad at Chess (low depths)... But that was 20 >>>>years ago ! Computer science evolved quite a bit since then. >>>> >>>>Do you like playing random tournaments or do you really want to get an accurate >>>>picture of the computer chess landscape ? >>> >>>Absolutely. The biggest problem (as I see it) is the quality of the games with >>>the longer time controls. After all, we won't see any brilliancies like these >>>at G/90 or slower on fast hardware: >>> >>>The true beauteous wonder that is "lightning chess" makes me pine for the days >>>when all games will be run at G/1 minute or faster. >> >>It's not that big of a problem IMO, you get a lot of games with G/5, so even if >>2 of 40 are flukes it won't matter, if you get 20 of 40 flukes then chances are >>they spread evenly between the two contenders, so again it won't matter. >>If you instead play two G/120 games, and one of them is a fluke, it matters a >>lot. >> >>In other words: Say I give you two hours, you need to figure out if an >>improvement you have made to your engine is really an improvement. >>Now do you play 1 or 2 slow games against a third opponent, or do you prefer 20 >>blitz games? >>I would _always_ pick the blitz gauntlet, for better statistics (more games + >>more opponents). >> >>I agree it is better (more interesting) to play 100 slow games than 100 fast >>games, but that is never the choice we have to make :) > >Here is a rating list of very quick paced games. You can find the actual games >here: >ftp://cap.connx.com/pub/The%20Gauntlet/calibration/quick/ >No set of contests with less than 20 games was included, to reduce the noise. > >If you think that the randomness evens out, think again. This is clearly wrong, >as a glance will tell you: > > > Program Elo + - Games Score Av.Op. Draws > > 1 Delfi 1.01 : 2799 88 502 20 95.0 % 2287 0.0 % > 2 LGoliath : 2757 71 332 22 95.5 % 2228 9.1 % > 3 Comet B.37 : 2724 113 370 20 92.5 % 2287 5.0 % > 4 Sjeng73 : 2672 38 187 122 89.8 % 2295 2.5 % > 5 MAD : 2664 111 391 21 90.5 % 2273 0.0 % > 6 Arasanx-54 : 2649 25 129 287 88.5 % 2294 1.4 % > 7 AnMon : 2649 40 121 129 83.3 % 2369 8.5 % > 8 Gully2 : 2629 107 391 22 90.9 % 2229 0.0 % > 9 Crafty-18.2 : 2625 111 300 21 90.5 % 2234 9.5 % > 10 Quark v1.46alpha : 2624 39 172 123 87.8 % 2281 1.6 % > 11 Morphy : 2623 102 320 24 89.6 % 2249 4.2 % > 12 Faile : 2598 101 273 25 88.0 % 2252 8.0 % > 13 Comet B.36 : 2594 28 114 247 84.6 % 2298 0.8 % > 14 Ghost : 2577 102 282 25 86.0 % 2262 4.0 % > 15 Gaviota : 2573 113 317 21 88.1 % 2226 4.8 % > 16 WildCat : 2552 118 315 20 87.5 % 2214 5.0 % > 17 Knightx : 2547 111 330 22 86.4 % 2227 0.0 % > 18 Sjeng 10 CCT : 2546 41 136 124 82.3 % 2280 1.6 % > 19 Yace 0.99.50 : 2545 33 109 189 81.7 % 2285 2.6 % > 20 TSCP-152 : 2542 44 110 115 80.4 % 2296 11.3 % > 21 Bestia : 2538 115 329 21 85.7 % 2227 0.0 % > 22 ExChess4 : 2537 106 281 24 85.4 % 2230 4.2 % > 23 Bringer-18 : 2532 26 81 306 79.2 % 2299 2.3 % > 24 TSCP-172 : 2529 44 125 115 79.6 % 2293 2.6 % > 25 Fortress : 2527 120 329 20 85.0 % 2226 0.0 % > 26 LG2000V3 : 2519 28 77 281 77.0 % 2308 2.5 % > 27 OliThink : 2515 120 265 20 85.0 % 2214 10.0 % > 28 GNU Chess v5.02 : 2513 27 76 305 77.5 % 2298 2.3 % > 29 Dragon-311 : 2511 28 81 286 77.4 % 2296 0.3 % > 30 Averno-30 : 2508 35 96 189 77.2 % 2295 1.1 % > 31 GNUChess4 : 2507 118 278 21 83.3 % 2227 4.8 % > 32 LordKingV2_9 : 2505 120 240 21 81.0 % 2254 9.5 % > 33 Beowulf : 2504 126 239 20 80.0 % 2263 10.0 % > 34 Phalanx-xxii : 2492 46 112 115 76.5 % 2287 3.5 % > 35 Bringer : 2491 121 220 22 77.3 % 2279 9.1 % > 36 Nejmet : 2482 126 288 20 80.0 % 2242 0.0 % > 37 TSCP : 2468 95 276 27 88.9 % 2107 7.4 % > 38 Ufim143 : 2467 48 102 115 72.6 % 2298 2.6 % > 39 Morphy-322 : 2459 43 85 148 70.6 % 2307 3.4 % > 40 Gully213p16 : 2459 30 62 293 71.0 % 2303 3.4 % > 41 ArasanX : 2451 118 228 23 76.1 % 2250 4.3 % > 42 LDBlanche : 2449 126 234 22 72.7 % 2279 0.0 % > 43 ExChess-402 : 2441 31 61 284 69.5 % 2297 2.5 % > 44 Holmes : 2440 128 248 20 77.5 % 2225 5.0 % > 45 Yace-020 : 2436 48 88 123 68.7 % 2300 4.1 % > 46 Averno-32 : 2429 32 57 285 67.4 % 2303 3.5 % > 47 Bestia-07 : 2427 33 58 271 67.0 % 2304 3.3 % > 48 Pepito v1.21 : 2414 48 90 124 68.5 % 2279 1.6 % > 49 King of Kings 1.95 : 2410 64 101 79 65.8 % 2296 5.1 % > 50 ZChess-222 : 2409 31 52 302 67.9 % 2279 9.9 % > 51 chezzz : 2406 126 189 23 69.6 % 2263 8.7 % > 52 WBQueen-214 : 2406 61 102 83 66.9 % 2284 3.6 % > 53 DrChess : 2404 129 209 22 70.5 % 2253 4.5 % > 54 WildCat-257 : 2402 53 80 116 63.8 % 2303 5.2 % > 55 LDBlanche-20c : 2400 33 53 290 63.3 % 2305 1.7 % > 56 DrChess-20 : 2394 34 53 284 63.4 % 2299 2.8 % > 57 ColChessp : 2393 135 157 21 69.0 % 2254 23.8 % > 58 chezzz-088 : 2391 34 51 283 63.4 % 2296 6.0 % > 59 AnMon-509 : 2390 36 53 262 61.3 % 2310 1.1 % > 60 ExChess-311 : 2390 34 51 284 62.9 % 2298 3.9 % > 61 Ghost-13 : 2388 34 51 295 62.0 % 2303 0.7 % > 62 AnMon-510 : 2387 35 53 268 61.6 % 2305 0.7 % > 63 FranWB : 2385 135 193 22 65.9 % 2271 4.5 % > 64 Ant : 2381 126 234 22 72.7 % 2211 0.0 % > 65 Mint v2.3 : 2374 34 46 306 60.1 % 2302 4.6 % > 66 OliThink-305c : 2368 55 80 118 61.0 % 2290 0.0 % > 67 Freyr : 2362 132 199 23 65.2 % 2253 0.0 % > 68 ZChess-120 : 2359 33 45 312 60.9 % 2282 6.4 % > 69 WinCraft : 2357 60 80 102 59.8 % 2288 3.9 % > 70 Gaviota-022 : 2355 36 46 286 57.5 % 2303 3.1 % > 71 Freyr-1067 : 2354 37 48 280 57.9 % 2299 1.4 % > 72 Beowulf-16b : 2354 30 38 433 57.3 % 2303 2.3 % > 73 Noonian-30 : 2351 57 71 120 58.3 % 2293 5.0 % > 74 LordKing-29 : 2347 37 44 288 55.9 % 2306 3.5 % > 75 DChess-102 : 2343 38 45 281 56.0 % 2300 3.2 % > 76 Esc : 2342 135 193 22 65.9 % 2228 4.5 % > 77 LordKing-29D : 2341 38 43 288 55.2 % 2305 4.9 % > 78 LarsenVB : 2335 139 165 23 60.9 % 2258 8.7 % > 79 Raffaela-005 : 2334 58 71 118 56.8 % 2287 3.4 % > 80 Green Light Chess v2.13: 2334 40 46 259 54.8 % 2300 3.1 % > 81 LambChop-710 : 2331 153 198 20 60.0 % 2261 0.0 % > 82 Esc-028 : 2326 39 44 278 54.1 % 2297 2.5 % > 83 Amyan-111 : 2324 39 39 298 52.3 % 2308 6.7 % > 84 Gerbil-01 : 2323 40 42 274 53.3 % 2300 5.1 % > 85 Noonian : 2322 142 160 22 61.4 % 2242 13.6 % > 86 King of Kings 1.96 : 2320 50 49 194 50.8 % 2315 4.6 % > 87 Aldebaran 0.7.0 : 2320 33 30 439 50.1 % 2319 9.1 % > 88 The Crazy Bishop, 46.01: 2315 78 92 68 57.4 % 2264 5.9 % > 89 Nejmet-260 : 2313 62 69 116 53.9 % 2286 0.9 % > 90 Storm-006 : 2311 62 67 118 52.5 % 2294 1.7 % > 91 DChess : 2311 139 174 24 58.3 % 2253 0.0 % > 92 Faile-14 : 2306 41 40 280 51.6 % 2295 6.8 % > 93 Amyan-1423 : 2297 44 43 245 48.4 % 2308 4.5 % > 94 SSEchess-2045 : 2292 61 64 120 49.6 % 2295 5.8 % > 95 Gerbil : 2283 150 159 23 54.3 % 2253 4.3 % > 96 EnginMax : 2281 153 184 21 57.1 % 2231 0.0 % > 97 Ant-606 : 2280 43 39 291 46.9 % 2301 1.0 % > 98 31337-30 : 2273 29 27 569 46.0 % 2301 7.9 % > 99 TCBishop-0045 : 2273 61 64 115 48.7 % 2282 8.7 % >100 Bace : 2268 55 34 260 33.7 % 2386 8.1 % >101 Holmes-040 : 2258 53 45 204 44.6 % 2295 2.0 % >102 Bace-45 : 2256 57 52 156 44.6 % 2294 7.1 % >103 Cilian : 2254 141 141 20 50.0 % 2254 30.0 % >104 Chessterfield : 2249 153 153 21 50.0 % 2249 14.3 % >105 LarsenVB-407 : 2244 45 35 291 40.9 % 2308 6.2 % >106 Amy : 2242 42 22 551 31.2 % 2379 6.2 % >107 Amyan : 2241 66 33 240 28.7 % 2399 6.7 % >108 Mustang-172 : 2240 68 56 121 41.7 % 2298 7.4 % >109 Gromit3 : 2230 44 34 302 40.2 % 2299 8.3 % >110 Betsy : 2226 172 172 20 50.0 % 2226 0.0 % >111 SSEchess-202 : 2226 69 57 116 40.5 % 2293 10.3 % >112 EnginMax-260 : 2222 53 35 272 38.4 % 2303 0.4 % >113 Chessterfield-15a : 2215 49 34 289 37.5 % 2304 6.6 % >114 Cilian-413 : 2212 42 34 279 37.3 % 2303 19.4 % >115 MAD-005 : 2209 51 33 285 36.5 % 2305 5.6 % >116 Blikskottel 0.5 : 2199 192 149 20 37.5 % 2287 5.0 % >117 Gromit2 : 2196 53 32 304 35.4 % 2300 1.6 % >118 FranWB-090 : 2191 53 32 287 34.3 % 2303 5.9 % >119 FRANWB-081 : 2189 54 32 283 33.4 % 2309 6.0 % >120 LarsenVB-313 : 2182 53 32 285 32.6 % 2308 8.4 % >121 Ant-416 : 2165 60 31 283 30.9 % 2304 3.2 % >122 MSCP : 2157 160 142 22 38.6 % 2238 13.6 % >123 Fortress-162 : 2153 62 30 284 29.6 % 2303 3.5 % >124 31337 : 2152 72 22 417 18.7 % 2407 4.8 % >125 NeroWB : 2137 168 167 20 47.5 % 2155 5.0 % >126 Pierre : 2131 182 124 25 34.0 % 2246 4.0 % >127 StAndersen : 2126 182 124 25 34.0 % 2241 4.0 % >128 Grizzly : 2125 192 142 21 35.7 % 2227 4.8 % >129 TCBishop-4601 : 2123 146 69 59 27.1 % 2294 0.0 % >130 Golem01 : 2119 68 28 298 25.8 % 2302 3.4 % >131 Ozwald-043 : 2106 110 44 125 24.4 % 2303 0.8 % >132 NeroWB-5 : 2096 115 44 122 23.4 % 2303 0.8 % >133 Knightx-154 : 2085 83 27 292 21.4 % 2311 0.3 % >134 Amy-07 : 2079 68 23 400 20.9 % 2310 4.8 % >135 InmiChess3 : 2033 95 25 305 17.5 % 2302 0.3 % >136 Chad's Chess V015 : 2031 107 31 214 18.0 % 2294 1.4 % >137 Zephyr-061 : 2027 105 32 203 18.7 % 2282 2.0 % >138 Tamerlane-01 : 1998 153 41 119 15.5 % 2292 0.8 % >139 ChadChess-015 : 1994 203 54 70 13.6 % 2315 1.4 % >140 Noonian-29 : 1975 159 40 121 14.0 % 2290 1.7 % >141 Betsy-526 : 1973 127 26 258 13.2 % 2300 0.0 % >142 VCTAO44 : 1969 164 41 116 14.7 % 2275 0.0 % >143 GNUChess-414 : 1966 126 25 281 12.5 % 2305 0.7 % >144 Grizzly-125 : 1964 123 25 276 12.3 % 2305 2.2 % >145 Rikus : 1957 330 111 22 13.6 % 2278 0.0 % >146 MSCP-011 : 1857 241 37 122 7.4 % 2297 0.0 % >147 aldebaran : 1836 0 0 100 0.0 % 2436 0.0 % >148 Aldebaran-070 : 1720 0 0 39 0.0 % 2320 0.0 % >149 Faile-06 : 1720 0 0 133 0.0 % 2320 0.0 % > >The speed of the game is directly proportional to the garbage introduced. >The games I posted were but a tiny sampling of the nonesense that is erupted >when you shrink the clock. > >IMO-YMMV The nonsense is mainly not because of the fact that you shrinked the clock but because of the fact that some program suck at very fast time control because they were not designed to play it(for example a program may count time in seconds and play immediately in 15 seconds per game). It does not contradict the fact that you can use time control of 15 second per game to test changes in the program if you expect them to help at all time controls and you have no bugs that prevent the program to search even to depth 1 at 15 seconds per game. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.