Author: Uri Blass
Date: 06:01:24 07/13/02
Go up one level in this thread
On July 13, 2002 at 08:28:52, José Carlos wrote: >On July 13, 2002 at 08:15:02, Uri Blass wrote: > >>On July 13, 2002 at 07:09:02, José Carlos wrote: >> >>>On July 13, 2002 at 05:35:24, Uri Blass wrote: >>> >>>>On July 12, 2002 at 19:16:31, José Carlos wrote: >>>> >>>>>On July 12, 2002 at 14:56:11, Ed Schröder wrote: >>>>> >>>>>>Hi CCC, >>>>>> >>>>>>In Rebel I maintain a statistic file, on every iteration a counter is >>>>>>incremented with 1 (see column 2) representing the iteration depths Rebel has >>>>>>searched. When a new best move is found a second counter is incremented with 1 >>>>>>(see column 3) representing how many times a new best move has been found on the >>>>>>given iteration depth, between brackets the percentage is calculated. >>>>>> >>>>>>As you can see the very first plies Rebel often changes to new best moves, >>>>>>however when the depth increases and increases the chance Rebel will change its >>>>>>mind drops and drops. From 16 plies on the chance a new better move is found is >>>>>>below 2%. >>>>>> >>>>>>I wonder what this all means, it is still said (and believed by many) that a >>>>>>doubling in computer speed gives 30-50-70 elo. That could be very well true for >>>>>>lower depths but the below statistic seem to imply something totally different, >>>>>>a sharp diminishing return on deeper depths. >>>>>> >>>>>>Interesting also is colum 4 (Big Score Changes), whenever a big score difference >>>>>>is measured (0.50 up or down) the percentage is calculated. This item seems to >>>>>>be less sensitive than the change in best move. However the maintained "Big >>>>>>Score Changes" statistic is not fully reliable as it also counts situations like >>>>>>being a rook or queen up (or down) in positions and naturally you get (too) many >>>>>>big score fluctuations. I have changed that and have limit the system to scores >>>>>>in the range of -2.50 / +2.50 but for the moment have too few games played to >>>>>>show the new statistic. >>>>>> >>>>>>Anyway the number of positions calculated seem to be more than sufficient (over >>>>>>100,000) to be reliable. The origin came from extensive testing the latest Rebel >>>>>>via self-play at various time controls. >>>>> >>>>> Hi Ed, if I get this right, the second column (moves searched) is the number >>>>>of positions in which the program has reached the depth given by column 1. If it >>>>>was really "moves", there would be about 3x in depth 2 than in depth 1. >>>>> Then the idea is that many more changes happen in low depths because the >>>>>program is there many more times, so I (ignoring "Big Changes") calculated a >>>>>couple of other numbers: >>>>> The ratio moves changes / moves searched and the relative % of changes from >>>>>ply to ply: >>>>> >>>>> SEARCH OVERVIEW >>>>> =============== >>>>> >>>>> (A) (B) (C) (D) (E) >>>>>Depth Moves Moves Moves Changed / rel % of changes from >>>>> Searched Changed Moves Searched ply n-1 to n >>>>> >>>>> 1 113768 0 = 0.0% 0 >>>>> 2 113768 44241 = 38.9% 0.388870333 >>>>> 3 113768 34262 = 30.1% 0.30115674 77.44% >>>>> 4 113194 32619 = 28.8% 0.288168984 95.69% >>>>> 5 113191 30697 = 27.1% 0.271196473 94.11% >>>>> 6 108633 28516 = 26.2% 0.262498504 96.79% >>>>> 7 108180 25437 = 23.5% 0.235135885 89.58% >>>>> 8 102782 22417 = 21.8% 0.218102391 92.76% >>>>> 9 82629 15400 = 18.6% 0.186375244 85.45% >>>>>10 59032 9144 = 15.5% 0.154899038 83.11% >>>>>11 39340 5183 = 13.2% 0.131748856 85.05% >>>>>12 23496 2350 = 10.0% 0.100017024 75.91% >>>>>13 12692 957 = 7.5% 0.075401828 75.39% >>>>>14 6911 396 = 5.7% 0.057299957 75.99% >>>>>15 4032 193 = 4.8% 0.047867063 83.54% >>>>>16 2471 72 = 2.9% 0.029138001 60.87% >>>>>17 1608 26 = 1.6% 0.016169154 55.49% >>>>>18 1138 17 = 1.5% 0.014938489 92.39% >>>>>19 921 6 = 0.7% 0.006514658 43.61% >>>>>20 795 7 = 0.9% 0.008805031 135.16% >>>>>21 711 1 = 0.1% 0.00140647 15.97% >>>>>22 636 2 = 0.3% 0.003144654 223.58% >>>>>23 574 5 = 0.9% 0.008710801 277.00% >>>>>24 507 1 = 0.2% 0.001972387 22.64% >>>>>25 451 3 = 0.7% 0.006651885 337.25% >>>>>26 394 1 = 0.3% 0.002538071 38.16% >>>>>27 343 2 = 0.6% 0.005830904 229.74% >>>>>28 296 2 = 0.7% 0.006756757 115.88% >>>>>29 269 0 = 0.0% 0 0.00% >>>>> >>>>> Column (D) means the probability at a certain position at a certain depth to >>>>>get a change, according to your data, for a random position (I assume you chose >>>>>random positions, because this data comes from real games). >>>> >>>>No >>>> >>>>I assume that the positions that was searched to big depthes like 16 are only >>>>positions that the program had enough time to search in the game to depth 16. >>>> >>>>These positions are not random positions from games. >>>>I expect in random positions from games to see at least 10% changes at depth 16. >>>> >>>>Uri >>> >>> It's interesting that Ed, who has been doing chess programming for a lot of >>>years rely on statistical data, and you, absolute newbie to chess programming >>>can 'expect'. Quite amazing. >>> >>> José C. >> >>I do not find it amazing. >> >>I have experience in correspondence games when I let programs to analyze for a >>lot of hours. >> >>The case when they change their mind to a new move after many hours is not rare >>and certainly happen in more than 2% and they get depthes that are bigger than >>16 after many hours. >> >>My estimate of 10-15% is based on my experience in correspondence games and >>reading material about the subject. >> >>I did not calculate statistics about deep searches for my correspondence games >>but I remember that I read researches when 13-14% at big depth was mentioned(I >>think it was about crafty goes deep and dark though go deep). >> >>Uri > > I don't think your correspondence games are a valid example because, if you >use some different programs and make the move one of them suggests, the others >are gonna have a different behaviour than with their own moves. For example, a >program can apply null move to every branch of the tree except the pv. Then, the >program refuted a move that you choose because of null move, and now it has to >search it without, so the hash entries are giving 'wrong' information. There are >more examples, like applying lazy eval. In summary, your correspondence analisys >don't show how the program would behave in a real game. However, it would be >interesting if you had some log files to examine here and try to draw >conclusions. > > José C. Here is some examples when deep fritz or shredder5.32 changed their mind at depth 16 or 17 after some hours of search(p450 or p850 was used) Uri Blass - Yoav dotan 2r2r1k/1bq1bpp1/p2ppn1p/1p2n3/3NP3/P1N2P1Q/1PP3PP/2BRRBK1 w - - 0 1 Analysis by Deep Fritz: 17.Qxe6-- -+ (-3.97) Depth: 1/3 00:00:00 17.Qxe6-- fxe6 -+ (-6.87) Depth: 1/4 00:00:00 17.Nxe6! -+ (-4.87) Depth: 1/4 00:00:00 17.Nxe6! fxe6 µ (-1.06) Depth: 1/4 00:00:00 17.Be3! = (0.25) Depth: 1/4 00:00:00 17.Be3 Kg8 18.f4 ² (0.28) Depth: 2/9 00:00:00 17.Be3 Kg8 18.f4 ² (0.28) Depth: 3/9 00:00:00 17.Be3 d5 18.f4 = (0.16) Depth: 4/12 00:00:00 17.Be3 d5 18.exd5 Nxd5 19.Nxd5 Bxd5 20.Qg3 = (0.06) Depth: 5/17 00:00:00 4kN 17.Bg5! = (0.09) Depth: 5/17 00:00:00 4kN 17.Bg5! Kg8 18.Be3 Rfd8 19.f4 Neg4 20.Bc1 e5 21.Nf5 Qc5+ 22.Be3 Nxe3 = (0.19) Depth: 5/17 00:00:00 5kN 17.Bg5 Kg8 18.Be3 Rfd8 19.Be2 = (0.25) Depth: 6/13 00:00:00 8kN 17.Bg5 Kg8 18.Be3 Rfe8 = (0.25) Depth: 7/26 00:00:00 105kN 17.Bg5 Rfd8 18.f4 Ned7 19.Bh4 Kg8 20.Bd3 d5 = (0.19) Depth: 8/32 00:00:02 581kN 17.Bg5 Rfd8 18.f4 Ned7 19.Be2 Kg8 20.Bh4 d5 21.e5 = (0.16) Depth: 9/32 00:00:06 1978kN 17.Bg5 Rfd8 18.Bf4 Nc4 19.Bxc4 bxc4 20.Be3 d5 21.e5 Nd7 22.f4 = (0.13) Depth: 10/38 00:00:25 7949kN 17.Qg3! = (0.16) Depth: 10/39 00:00:32 10062kN 17.Qg3 Rfd8 18.Be3 Ned7 19.Bd3 d5 20.Qxc7 Rxc7 21.exd5 Nxd5 22.Nxd5 Bxd5 = (0.09) Depth: 11/33 00:00:47 14759kN 17.Qg3 Nh5 18.Qf2 Nf6 19.f4 Nc4 20.Bxc4 Qxc4 = (0.09) Depth: 12/38 00:02:18 44669kN 17.Be3! = (0.13) Depth: 12/38 00:02:50 54772kN 17.Be3 Nc4 18.Bxc4 bxc4 19.Qg3 Rb8 20.Nde2 Ba8 21.Rb1 Rb7 22.b3 cxb3 = (0.06) Depth: 13/39 00:04:56 96523kN 17.Bg5! = (0.09) Depth: 13/40 00:07:43 151949kN 17.Bg5 Rfd8 18.f4 Nc6 19.Nxc6 Qxc6 20.f5 d5 21.exd5 Bc5+ 22.Be3 exd5 = (0.06) Depth: 14/42 00:17:32 345585kN 17.Bg5 Rfd8 18.f4 Qc5 19.Kh1 Neg4 20.Bh4 Kg8 21.Re2 Qh5 22.Be1 = (0.03) Depth: 15/44 01:00:14 1195720kN 17.f4! = (0.06) Depth: 15/45 02:07:49 2543835kN 17.f4 Qb6 18.Qg3 Ned7 19.e5 dxe5 20.fxe5 Nd5 21.Nxd5 Bxd5 22.c3 Qd8 = (0.00) Depth: 16/45 03:18:59 3978273kN 17.Bg5! = (0.03) Depth: 16/45 04:19:35 5179149kN (blass, telaviv 29.04.2001) Uri Blass - Yoav dotan [D]2r3r1/1bqnbp1k/p2pp3/1p4p1/2nNP2p/P1N2P1P/1PP2QP1/2BRRB1K b - - 0 1 Analysis by Deep Fritz: 24...Bxe4-- +- (1.78) Depth: 1/3 00:00:00 24...Bxe4-- 25.fxe4 +- (2.66) Depth: 1/3 00:00:00 24...Nxb2! +- (2.53) Depth: 1/3 00:00:00 24...Nxa3! +- (2.34) Depth: 1/3 00:00:00 24...Nxa3! = (-0.25) Depth: 1/4 00:00:00 24...Nxa3 25.Be3 = (-0.06) Depth: 2/7 00:00:00 24...Nxa3 25.Be3 Ne5 = (-0.22) Depth: 3/14 00:00:00 24...Nxa3 25.Bd3 Ne5 26.Be3 = (-0.16) Depth: 4/15 00:00:00 2kN 24...Nxa3 25.Bd3 Ne5 26.Be3 Nac4 27.Bxc4 ³ (-0.38) Depth: 5/25 00:00:00 19kN 24...Nxa3 25.Be3 Ne5 26.Rd2 Nac4 27.Bxc4 Nxc4 ³ (-0.38) Depth: 6/31 00:00:00 96kN 24...Nxa3 25.Bd2 Nc4 26.Bxc4 Qxc4 27.Be3 b4 28.Nce2 ³ (-0.41) Depth: 7/31 00:00:00 342kN 24...Nxa3 25.Nd5 exd5 26.exd5 Ne5 27.bxa3 Rge8 28.a4 bxa4 ³ (-0.34) Depth: 8/35 00:00:01 603kN 24...Nxa3 25.Nd5 exd5 26.exd5 Ne5 27.bxa3 Rge8 28.a4 bxa4 ³ (-0.34) Depth: 9/36 00:00:02 1444kN 24...Nxa3 25.Nd5 exd5 26.exd5 Nb6 27.Nf5 Bf6 28.bxa3 Nxd5 29.Bd3 ³ (-0.38) Depth: 10/38 00:00:05 3625kN 24...Nxa3 25.Nd5 exd5 26.exd5 Nb6 27.bxa3 Nxd5 28.Bd3+ Kh8 29.Bf5 Nc3 30.Bxc8 ³ (-0.34) Depth: 11/38 00:00:16 10497kN 24...Nxa3 25.Nd5 exd5 26.exd5 Nb6 27.bxa3 Bf6 28.a4 bxa4 29.Bd3+ Kh8 30.Bf5 = (-0.19) Depth: 12/41 00:00:34 22616kN 24...Nxa3 25.Nd5 exd5 26.exd5 Nb6 27.bxa3 Nxd5 28.Bd3+ Kh8 29.Nf5 f6 30.Qd4 = (0.03) Depth: 13/41 00:01:28 60293kN 24...Nxa3 25.Nd5 exd5 26.exd5 Bf6 27.bxa3 Qc5 28.Bd3+ Kh8 29.Bf5 Rgd8 30.f4 = (0.09) Depth: 14/46 00:04:26 183778kN 24...Nxa3 25.Nd5 exd5 26.exd5 Nb6 27.bxa3 Nxd5 28.Bd3+ Kh8 29.Nf5 Bf6 30.Be4 = (0.00) Depth: 15/46 00:15:20 587191kN 24...Nxa3 25.Nd5 exd5 26.exd5 Bf6 27.bxa3 Rcd8 28.Bd3+ Kh8 29.Bf5 Rge8 30.Rxe8+ ² (0.34) Depth: 16/49 01:01:58 1966300kN 24...Nxa3 25.f4 g4 26.f5 e5 27.f6 Bxf6 28.Nf5 Nc4 29.Bxc4 ² (0.56) Depth: 17/50 08:06:11 16611218kN 24...Nf8! ² (0.53) Depth: 17/50 09:38:18 20439760kN 24...Nf8! 25.Bxc4 bxc4 26.Be3 Ng6 27.Nde2 Bc6 28.Rb1 Rb8 29.Nd4 Bf6 ² (0.47) Depth: 17/53 11:01:54 23930252kN 24...Nde5! ² (0.44) Depth: 17/53 12:38:50 27916197kN Uri Blass - Yoav dotan [D]2r3r1/1b1nbp1k/p2pp3/1p4p1/2qNP2p/P1N1BP1P/1PP2QP1/3RR2K w - - 0 1 Analysis by Shredder 5.32: 25.Ndxb5 axb5 26.b3 Qc6 27.Kg1 ³ (-0.32) Depth: 1/2 00:00:00 25.Ndxb5 axb5 26.b3 Qc6 27.Kg1 µ (-1.27) Depth: 1/2 00:00:00 25.Ndxb5 axb5 26.b3 Qc6 27.Kg1 -+ (-2.56) Depth: 1/2 00:00:00 25.Kh2 Rg6 26.Nb3 Qc7 27.Bd4 Kg8 µ (-1.24) Depth: 1/2 00:00:00 25.Kh2 Rg6 26.Nb3 Qc7 27.Bd4 Kg8 = (0.08) Depth: 1/2 00:00:00 25.Kh2 Rg6 26.Nb3 Qc7 27.Bd4 Kg8 = (-0.10) Depth: 1/2 00:00:00 25.Kh2 Bxe4 26.fxe4 = (-0.09) Depth: 2/4 00:00:00 25.Kh2 f6 26.f4 Kg7 27.fxg5 = (-0.10) Depth: 3/6 00:00:00 25.f4 Ne5 26.fxg5 Bxg5 27.Bxg5 Rgd8 = (-0.09) Depth: 3/6 00:00:00 25.f4 f6 26.g4 = (-0.10) Depth: 3/6 00:00:00 25.f4 g4 26.hxg4 Rxg4 27.Bd2 = (-0.13) Depth: 4/8 00:00:00 25.Kh2 Bxe4 26.Nxe4 e5 = (-0.12) Depth: 4/8 00:00:00 25.Kh2 f6 26.a4 bxa4 27.Nd5 = (-0.09) Depth: 4/8 00:00:00 25.Kh2 f6 26.Qf1 Qxf1 27.Rxf1 Nc5 = (-0.11) Depth: 5/10 00:00:00 25.f4 g4 26.hxg4 Rxg4 = (-0.10) Depth: 5/10 00:00:00 25.f4 g4 26.Qf3 gxh3 27.gxh3 Kg7 28.Ndxb5 = (-0.10) Depth: 5/10 00:00:00 25.f4 g4 26.hxg4 Rxg4 27.a4 Nf6 28.Bd2 ³ (-0.29) Depth: 6/12 00:00:00 22kN 25.Kh2 f6 26.Qf1 Bxe4 27.Nxe4 Qxf1 28.Rxf1 ³ (-0.28) Depth: 6/12 00:00:00 25kN 25.Kh2 Rg6 26.Nb3 Qc7 27.Bd4 e5 28.Be3 = (-0.10) Depth: 6/12 00:00:00 43kN 25.Kh2 Bf6 26.Kg1 Kh8 27.Nde2 Be7 28.Bf4 = (-0.09) Depth: 7/14 00:00:00 98kN 25.Kh2 Bf6 26.Kg1 b4 27.Nb1 bxa3 28.Nxa3 Qa2 29.c3 = (-0.16) Depth: 8/16 00:00:01 203kN 25.Rf1 f6 26.f4 gxf4 27.Qxf4 = (-0.15) Depth: 8/16 00:00:02 426kN 25.Rf1 Bf6 26.Nde2 b4 27.b3 Qc7 28.Na2 bxa3 = (-0.14) Depth: 8/16 00:00:04 741kN 25.Nb3 Qc7 = (-0.11) Depth: 8/16 00:00:05 841kN 25.Nb3 Qc7 = (-0.11) Depth: 8/16 00:00:05 842kN 25.Kg1 Kg7 26.Nb3 Qxc3 27.Bd4+ e5 28.bxc3 exd4 = (-0.10) Depth: 8/16 00:00:06 996kN 25.Kg1 Kh8 26.Nb3 Qc7 27.Bb6 Nxb6 = (-0.10) Depth: 8/16 00:00:06 1079kN 25.Kg1 Rg6 26.Nb3 Qc7 27.Bd4 Kg8 28.a4 bxa4 29.Nd2 = (-0.06) Depth: 9/18 00:00:11 1986kN 25.Kg1 Kh8 26.Nb3 Qc7 27.Qe2 Kg7 28.Kf2 f6 29.a4 = (-0.07) Depth: 10/20 00:00:18 3139kN 25.Kg1 Kh8 26.Nb3 Qc7 27.Qd2 = (-0.11) Depth: 11/22 00:00:35 6019kN 25.Kg1 Rg6 26.Nb3 Qc7 27.Bd4 Kg8 28.Nd2 Ne5 29.Qe3 Bf6 = (-0.06) Depth: 12/24 00:03:12 32396kN 25.Kg1 Rg6 26.Qd2 Ne5 27.Nde2 Qc7 28.b3 Qa5 29.Qd4 Nc6 = (-0.16) Depth: 13/26 00:07:39 77586kN 25.Nde2 Kg7 26.f4 Bxe4 27.Bd4+ Ne5 = (-0.15) Depth: 13/26 00:12:29 126346kN 25.Nde2 Qc7 26.Kg1 Kh8 27.f4 gxf4 28.Bd4+ Ne5 29.Kh2 Bf6 30.Qxf4 = (-0.15) Depth: 13/26 00:18:33 186562kN 25.Qd2 f6 26.Nb3 Qc7 27.Nd4 Kg7 28.Nxe6+ Kg6 = (-0.14) Depth: 13/26 00:27:00 271510kN 25.Qd2 Ne5 26.Qf2 Qc7 27.Bxg5 Rxg5 28.f4 Rg7 29.fxe5 dxe5 = (-0.14) Depth: 13/26 00:28:56 290256kN 25.Qd2 Ne5 26.Nde2 f6 27.b3 Qc6 28.Bd4 Rgf8 29.Kh2 Kg8 30.f4 ³ (-0.27) Depth: 14/28 00:48:58 493251kN 25.Nde2 Qc7 26.Kg1 Kh8 27.f4 gxf4 28.Nxf4 Ne5 29.Bb6 Qc4 30.Bd4 b4 ³ (-0.26) Depth: 14/28 00:51:01 513446kN 25.Nde2 Qc7 26.Kg1 Ne5 27.Bb6 Qd7 28.Bd4 Nc4 29.Nb1 f6 30.b3 = (-0.13) Depth: 14/28 00:59:52 600612kN 25.Kg1 Rg6 26.Nb3 Qc7 27.Bd4 Kg8 = (-0.12) Depth: 14/28 01:07:03 672277kN 25.Kg1 Rg6 26.Nb3 Qc7 27.Bd4 Kg8 28.a4 b4 29.f4 bxc3 30.Bb6 = (-0.12) Depth: 14/28 01:08:42 689036kN 25.Kg1 Rg6 26.Re2 Kg8 27.Nb3 Qc7 28.Red2 Rf8 29.Nxb5 = (-0.18) Depth: 15/30 01:56:24 1170440kN 25.Nde2 Qc7 26.Kg1 Ne5 27.Bb6 Qd7 28.Bd4 Nc4 29.Nb1 f6 30.b3 = (-0.17) Depth: 15/30 02:02:35 1230887kN 25.Nde2 Qc7 26.Rf1 Rg7 27.Bd4 f6 28.f4 e5 29.fxe5 fxe5 30.Ba7 Rf8 = (-0.14) Depth: 15/30 02:27:10 1473684kN 25.Re2 g4 26.fxg4 Kh8 = (-0.13) Depth: 15/30 03:41:46 2234563kN 25.Re2 Rg6 26.Red2 Rd8 27.Ndxb5 axb5 28.Rd4 Qc7 29.Rxd6 Qxd6 30.Rxd6 = (-0.13) Depth: 15/30 04:34:17 2769639kN 25.Nb3 Qc7 26.Bd4 Rge8 27.f4 Kg8 28.fxg5 Bxg5 29.Rf1 Rf8 30.Be3 Bf6 = (-0.12) Depth: 15/30 04:39:13 2818731kN 25.Nb3 Qc7 26.Bd4 Nc5 27.Nd2 Rg6 28.Kg1 Kg8 29.Nf1 f6 30.Ne3 = (-0.09) Depth: 15/30 04:48:01 2905480kN 25.Nb3 Qc7 26.Bd4 f6 27.f4 e5 28.fxg5 fxg5 29.Qf5+ Rg6 30.Nd5 exd4 31.Nxe7 d5 = (-0.15) Depth: 16/32 06:58:35 4263862kN 25.Re2 Rg6 26.Red2 Rd8 27.Nb3 Qc7 28.Bd4 = (-0.14) Depth: 16/32 07:56:33 4852245kN (, eim 03.07.2001) Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.