Author: Vincent Diepeveen
Date: 05:51:25 02/01/01
Go up one level in this thread
On February 01, 2001 at 04:18:42, Andrew Dados wrote: Your model applies to human games, not for prepared computerchess books which have played thousands of games to just beat another program which always plays the same... If you send me a version of whatever program, and knowing that all programs have very small tournament books, i can simply play 1.e4 till i found lines to beat it. You can only play 1.e4 against the strongest commercials if you have figured out your lines very well, as the theory is always on the move and from intereesting lines the judgement one week to another changes from --++ to ++-- and anything in that domain the week after this. *any* statistical model is taking into account that the statistical model applies for every game. So computer world championships and other tournaments where the programmers prepare for without the other side knowing what they play, this model applies. Note not all programmers do something during a tournament. Some have an openingsbook and also play every other game the same lines as they previously lost in such a tournament. Very well known is also how book learning of most work. Most operators (not referring to programmers who in general are a bit smarter) are that ignorant that they don't even turn off book learning with as result that the next same color game it will repeat the same line, if it won the first game with that line one of the previous rounds. But the bottom line is that for home tested engines against existing commercial products with a certain openingsbook it's pretty easy to cook for that. The better a book is from a certain program, the harder of course it is to find wins against mainlines such a program plays. In general the statistical model applies to mankind especially as they learn after each game (the strong players certainly do), something no program does. If they have learning then this is a very primitif form of learning which is completely understood by competitors and taken into account. >After reading all those posts about statistical significance I decided to run >some tests. >My model is based on win expectancy formula: This assumption is entirely true for games between chessplayers, but for chessprograms that play exactly the same as a cdrom plays of it which i have here at home we talk about different things. It is simply a graph where i need to find a single path in, now i simply close path by path till there is no solution for the graph to win from me! >D:=1/(1+exp(delta/400)) where delta is rating difference. > >For a single match I started with calculating probabilities of 3 events: win, >draw and loss (p1,pd and p0). >Note you have freedom to set draw probability within those limits: > > D = p1 + 0.5*pd and > 1-D = p0 + 0.5*pd; > >I started with initial probability of draw set for 'equal opponents' (delta=0) >then scaled it down by some factor depending on delta. > >Then all you need is enumerate through all posible scores and all games.... >For equal opponents (delta=0) and match of 5 games you can get (score of 1/2 >scaled to 1): > >p1/p0/pd = 0.250 0.250 0.500 (50% draws): > >score probability >0-->0.098 >1-->0.977 >2-->4.395 >3-->11.719 >4-->20.508 >5-->24.609 >6-->20.508 >7-->11.719 <- means: in 5 game match you'll get 3.5 pts with 11.7% chance >8-->4.395 >9-->0.977 >10-->0.098 > >p1/p0/pd = 0.350 0.350 0.300 (30% draws): > >0-->0.525 >1-->2.251 >2-->6.485 >3-->12.311 >4-->18.246 >5-->20.364 >6-->18.246 >7-->12.311 <- less draws; bigger chance of getting 3.5 pts... >8-->6.485 >9-->2.251 >10-->0.525 > >p1/p0/pd = 0.500 0.500 0.000 (no draws model): > >0-->3.125 >1-->0.000 >2-->15.625 >3-->0.000 >4-->31.250 >5-->0.000 >6-->31.250 >7-->0.000 >8-->15.625 >9-->0.000 >10-->3.125 > >Note big impact of draw chance on resulting model. > > > >Now big tables... > >**** >Explanation: in below table entry marked < > means: > >"For match of 30 games against opponent rated 40 elo better then me >I have 22.20% chances to get 50% score or better." > > >**** > >============================================= >1) Expectancy of getting 50% score or better: >============================================== > >(Draw probability for equal opponents: 50%) >----------------------------------------------------------------------------- > 200 160 120 80 40 0 -40 -80 -120 <-delta >games: ---------------------------------------------------------------------- > 5: 98.38 96.30 92.29 85.49 75.38 62.30 47.66 33.43 21.42 12.54 > 10: 99.72 98.86 96.19 89.65 77.14 58.81 38.29 20.77 9.28 3.41 > 15: 99.95 99.64 98.09 92.63 79.31 57.22 32.53 13.93 4.41 1.03 > 20: 99.99 99.88 99.03 94.71 81.32 56.27 28.30 9.67 2.18 0.33 > 25: 100.00 99.96 99.50 96.17 83.11 55.61 24.96 6.85 1.11 0.11 > 30: 100.00 99.99 99.74 97.21 84.70 55.13 <22.20> 4.91 0.57 0.03 > 35: 100.00 100.00 99.86 97.96 86.11 54.75 19.87 3.55 0.30 0.01 > 40: 100.00 100.00 99.93 98.50 87.36 54.45 17.88 2.59 0.16 0.00 > 45: 100.00 100.00 99.96 98.89 88.48 54.19 16.14 1.90 0.08 0.00 > 50: 100.00 100.00 99.98 99.18 89.49 53.98 14.62 1.40 0.04 0.00 >probability of: ------------------------------------------------------------- >win : 0.58 0.51 0.44 0.38 0.31 0.25 0.20 0.15 0.11 0.08 >draw : 0.37 0.41 0.44 0.47 0.49 0.50 0.49 0.47 0.44 0.41 >loss : 0.06 0.08 0.11 0.15 0.20 0.25 0.31 0.38 0.44 0.51 > > >(Draw probability for equal opponents: 30%) >----------------------------------------------------------------------------- > 200 160 120 80 40 0 -40 -80 -120 <-delta >games: ---------------------------------------------------------------------- > 5: 96.25 93.20 88.33 81.19 71.66 60.18 47.74 35.63 24.99 16.51 > 10: 98.97 97.19 93.21 85.60 73.44 57.39 40.00 24.54 13.20 6.25 > 15: 99.70 98.79 95.94 88.90 75.47 56.08 35.06 18.00 7.53 2.58 > 20: 99.91 99.47 97.54 91.37 77.37 55.28 31.37 13.61 4.46 1.11 > 25: 99.97 99.76 98.49 93.24 79.08 54.73 28.38 10.46 2.69 0.49 > 30: 99.99 99.89 99.06 94.67 80.63 54.32 25.88 8.14 1.65 0.22 > 35: 100.00 99.95 99.41 95.77 82.03 54.01 23.73 6.38 1.02 0.10 > 40: 100.00 99.98 99.63 96.64 83.30 53.75 21.84 5.04 0.64 0.05 > 45: 100.00 99.99 99.77 97.31 84.45 53.54 20.17 3.99 0.40 0.02 > 50: 100.00 100.00 99.85 97.85 85.50 53.36 18.68 3.18 0.25 0.01 >probability of: ------------------------------------------------------------- >win : 0.65 0.59 0.53 0.47 0.41 0.35 0.29 0.24 0.20 0.16 >draw : 0.22 0.24 0.27 0.28 0.30 0.30 0.30 0.28 0.27 0.24 >loss : 0.13 0.16 0.20 0.24 0.29 0.35 0.41 0.47 0.53 0.59 > > > >(Draw probability for equal opponents: 10%) >----------------------------------------------------------------------------- > 200 160 120 80 40 0 -40 -80 -120 <-delta >games: ---------------------------------------------------------------------- > 5: 93.42 89.43 83.76 76.23 66.94 56.35 45.27 34.59 25.13 17.37 > 10: 98.10 95.63 90.91 83.00 71.49 57.12 41.75 27.68 16.60 9.03 > 15: 99.25 97.67 93.78 85.84 72.69 55.18 36.57 20.81 10.10 4.19 > 20: 99.72 98.82 95.90 88.59 74.65 54.69 33.46 16.65 6.67 2.16 > 25: 99.89 99.37 97.22 90.63 76.22 54.15 30.69 13.37 4.42 1.11 > 30: 99.96 99.67 98.11 92.28 77.71 53.81 28.41 10.89 2.98 0.58 > 35: 99.98 99.82 98.70 93.61 79.05 53.53 26.40 8.93 2.03 0.31 > 40: 99.99 99.90 99.11 94.69 80.29 53.31 24.64 7.37 1.39 0.17 > 45: 100.00 99.95 99.38 95.57 81.42 53.12 23.06 6.10 0.96 0.09 > 50: 100.00 99.97 99.57 96.30 82.47 52.96 21.63 5.08 0.66 0.05 >probability of: ------------------------------------------------------------- >win : 0.72 0.67 0.62 0.57 0.51 0.45 0.39 0.34 0.29 0.24 >draw : 0.07 0.08 0.09 0.09 0.10 0.10 0.10 0.09 0.09 0.08 >loss : 0.20 0.24 0.29 0.34 0.39 0.45 0.51 0.57 0.62 0.67 > > > >============================================= >2) Expectancy of getting 60% score or better: >============================================== > > >(Draw probability for equal opponents: 50%) >----------------------------------------------------------------------------- > 200 160 120 80 40 0 -40 -80 -120 <-delta >games: ---------------------------------------------------------------------- > 5: 93.27 87.46 78.58 66.57 52.34 37.70 24.62 14.51 7.71 3.70 > 10: 96.78 91.41 80.80 64.22 44.04 25.17 11.71 4.38 1.31 0.32 > 15: 98.44 94.15 83.24 63.57 38.99 18.08 6.09 1.46 0.25 0.03 > 20: 99.23 95.98 85.39 63.43 35.26 13.41 3.30 0.51 0.05 0.00 > 25: 99.62 97.21 87.24 63.50 32.26 10.13 1.83 0.18 0.01 0.00 > 30: 99.81 98.06 88.82 63.68 29.75 7.75 1.03 0.07 0.00 0.00 > 35: 99.90 98.64 90.18 63.91 27.58 5.98 0.59 0.02 0.00 0.00 > 40: 99.95 99.04 91.36 64.18 25.68 4.65 0.34 0.01 0.00 0.00 > 45: 99.97 99.32 92.38 64.47 23.98 3.63 0.19 0.00 0.00 0.00 > 50: 99.99 99.52 93.27 64.76 22.46 2.84 0.11 0.00 0.00 0.00 >probability of: ------------------------------------------------------------- >win : 0.58 0.51 0.44 0.38 0.31 0.25 0.20 0.15 0.11 0.08 >draw : 0.37 0.41 0.44 0.47 0.49 0.50 0.49 0.47 0.44 0.41 >loss : 0.06 0.08 0.11 0.15 0.20 0.25 0.31 0.38 0.44 0.51 > > > >(Draw probability for equal opponents: 30%) >----------------------------------------------------------------------------- > 200 160 120 80 40 0 -40 -80 -120 <-delta >games: ---------------------------------------------------------------------- > 5: 89.67 83.49 75.01 64.37 52.26 39.82 28.34 18.81 11.67 6.80 > 10: 94.02 87.60 77.03 62.30 45.18 28.77 15.88 7.56 3.12 1.12 > 15: 96.51 90.72 79.33 61.72 40.85 22.18 9.65 3.34 0.92 0.21 > 20: 97.93 93.00 81.41 61.58 37.62 17.58 6.08 1.54 0.29 0.04 > 25: 98.75 94.68 83.25 61.63 35.01 14.16 3.91 0.72 0.09 0.01 > 30: 99.24 95.93 84.86 61.77 32.79 11.54 2.55 0.35 0.03 0.00 > 35: 99.54 96.87 86.29 61.97 30.87 9.47 1.68 0.17 0.01 0.00 > 40: 99.72 97.58 87.56 62.19 29.15 7.82 1.11 0.08 0.00 0.00 > 45: 99.83 98.13 88.70 62.42 27.61 6.49 0.74 0.04 0.00 0.00 > 50: 99.89 98.54 89.71 62.67 26.21 5.40 0.50 0.02 0.00 0.00 >probability of: ------------------------------------------------------------- >win : 0.65 0.59 0.53 0.47 0.41 0.35 0.29 0.24 0.20 0.16 >draw : 0.22 0.24 0.27 0.28 0.30 0.30 0.30 0.28 0.27 0.24 >loss : 0.13 0.16 0.20 0.24 0.29 0.35 0.41 0.47 0.53 0.59 > > >(Draw probability for equal opponents: 10%) >----------------------------------------------------------------------------- > 200 160 120 80 40 0 -40 -80 -120 <-delta >games: ---------------------------------------------------------------------- > 5: 88.55 82.63 74.87 65.41 54.73 43.65 33.06 23.77 16.24 10.57 > 10: 92.00 85.19 75.00 61.65 46.50 31.73 19.41 10.60 5.18 2.27 > 15: 94.69 88.08 76.73 60.65 42.19 25.18 12.68 5.35 1.90 0.57 > 20: 96.49 90.47 78.59 60.39 39.18 20.67 8.65 2.84 0.74 0.15 > 25: 97.67 92.36 80.33 60.39 36.80 17.27 6.04 1.55 0.29 0.04 > 30: 98.44 93.85 81.91 60.50 34.79 14.58 4.27 0.86 0.12 0.01 > 35: 98.95 95.03 83.33 60.66 33.04 12.40 3.05 0.48 0.05 0.00 > 40: 99.29 95.97 84.62 60.85 31.48 10.60 2.19 0.27 0.02 0.00 > 45: 99.52 96.72 85.78 61.06 30.07 9.11 1.59 0.15 0.01 0.00 > 50: 99.67 97.32 86.83 61.28 28.78 7.85 1.15 0.09 0.00 0.00 >probability of: ------------------------------------------------------------- >win : 0.72 0.67 0.62 0.57 0.51 0.45 0.39 0.34 0.29 0.24 >draw : 0.07 0.08 0.09 0.09 0.10 0.10 0.10 0.09 0.09 0.08 >loss : 0.20 0.24 0.29 0.34 0.39 0.45 0.51 0.57 0.62 0.67 > > > >============================================= >3) Expectancy of getting 75% score or better: >============================================== > > >(Draw probability for equal opponents: 50%) >----------------------------------------------------------------------------- > 200 160 120 80 40 0 -40 -80 -120 <-delta >games: ---------------------------------------------------------------------- > 5: 79.82 68.98 55.78 41.57 28.18 17.19 9.38 4.56 1.98 0.78 > 10: 65.63 47.61 29.55 15.19 6.30 2.07 0.53 0.11 0.02 0.00 > 15: 71.82 50.50 28.39 12.08 3.72 0.81 0.12 0.01 0.00 0.00 > 20: 63.96 38.67 16.97 5.03 0.95 0.11 0.01 0.00 0.00 0.00 > 25: 69.62 41.72 16.95 4.23 0.60 0.05 0.00 0.00 0.00 0.00 > 30: 63.71 33.13 10.55 1.84 0.16 0.01 0.00 0.00 0.00 0.00 > 35: 68.76 35.85 10.69 1.58 0.10 0.00 0.00 0.00 0.00 0.00 > 40: 63.86 29.04 6.80 0.70 0.03 0.00 0.00 0.00 0.00 0.00 > 45: 68.43 31.43 6.94 0.61 0.02 0.00 0.00 0.00 0.00 0.00 > 50: 64.16 25.79 4.48 0.27 0.01 0.00 0.00 0.00 0.00 0.00 >probability of: ------------------------------------------------------------- >win : 0.58 0.51 0.44 0.38 0.31 0.25 0.20 0.15 0.11 0.08 >draw : 0.37 0.41 0.44 0.47 0.49 0.50 0.49 0.47 0.44 0.41 >loss : 0.06 0.08 0.11 0.15 0.20 0.25 0.31 0.38 0.44 0.51 > > > > >(Draw probability for equal opponents: 30%) >----------------------------------------------------------------------------- > 200 160 120 80 40 0 -40 -80 -120 <-delta >games: ---------------------------------------------------------------------- > 5: 76.26 66.67 55.44 43.41 31.75 21.57 13.58 7.92 4.30 2.19 > 10: 63.98 48.72 33.11 19.67 10.04 4.36 1.61 0.51 0.14 0.03 > 15: 69.19 50.99 31.94 16.40 6.71 2.15 0.54 0.10 0.02 0.00 > 20: 62.39 40.91 21.30 8.39 2.41 0.50 0.07 0.01 0.00 0.00 > 25: 67.20 43.45 21.24 7.35 1.71 0.26 0.03 0.00 0.00 0.00 > 30: 62.09 36.03 14.72 3.92 0.64 0.06 0.00 0.00 0.00 0.00 > 35: 66.39 38.35 14.84 3.49 0.47 0.03 0.00 0.00 0.00 0.00 > 40: 62.16 32.38 10.50 1.90 0.18 0.01 0.00 0.00 0.00 0.00 > 45: 66.06 34.46 10.65 1.72 0.13 0.00 0.00 0.00 0.00 0.00 > 50: 62.38 29.43 7.64 0.95 0.05 0.00 0.00 0.00 0.00 0.00 >probability of: ------------------------------------------------------------- >win : 0.65 0.59 0.53 0.47 0.41 0.35 0.29 0.24 0.20 0.16 >draw : 0.22 0.24 0.27 0.28 0.30 0.30 0.30 0.28 0.27 0.24 >loss : 0.13 0.16 0.20 0.24 0.29 0.35 0.41 0.47 0.53 0.59 > > > >(Draw probability for equal opponents: 10%) >----------------------------------------------------------------------------- > 200 160 120 80 40 0 -40 -80 -120 <-delta >games: ---------------------------------------------------------------------- > 5: 71.13 62.13 52.08 41.59 31.46 22.44 15.05 9.48 5.63 3.15 > 10: 61.58 48.14 34.44 22.20 12.74 6.45 2.87 1.12 0.39 0.12 > 15: 67.70 51.49 34.49 19.83 9.58 3.84 1.27 0.34 0.08 0.01 > 20: 61.38 42.37 24.47 11.42 4.19 1.19 0.26 0.04 0.01 0.00 > 25: 65.44 44.40 24.25 10.17 3.16 0.71 0.11 0.01 0.00 0.00 > 30: 60.89 37.83 17.94 6.13 1.45 0.23 0.02 0.00 0.00 0.00 > 35: 64.75 39.90 18.05 5.59 1.12 0.14 0.01 0.00 0.00 0.00 > 40: 60.96 34.54 13.60 3.44 0.53 0.05 0.00 0.00 0.00 0.00 > 45: 64.43 36.39 13.74 3.16 0.41 0.03 0.00 0.00 0.00 0.00 > 50: 61.13 31.84 10.48 1.97 0.20 0.01 0.00 0.00 0.00 0.00 >probability of: ------------------------------------------------------------- >win : 0.72 0.67 0.62 0.57 0.51 0.45 0.39 0.34 0.29 0.24 >draw : 0.07 0.08 0.09 0.09 0.10 0.10 0.10 0.09 0.09 0.08 >loss : 0.20 0.24 0.29 0.34 0.39 0.45 0.51 0.57 0.62 0.67 > > > >============================================== >3) Expectancy of getting 100% score: >============================================== > > > >(Draw probability for equal opponents: 50%) >----------------------------------------------------------------------------- > 200 160 120 80 40 0 -40 -80 -120 <-delta >games: ---------------------------------------------------------------------- > 5: 6.41 3.50 1.72 0.75 0.29 0.10 0.03 0.01 0.00 0.00 > 10: 0.41 0.12 0.03 0.01 0.00 0.00 0.00 0.00 0.00 0.00 > 15: 0.03 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 > 20: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 > 25: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 > 30: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 > 35: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 > 40: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 > 45: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 > 50: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 >probability of: ------------------------------------------------------------- >win : 0.58 0.51 0.44 0.38 0.31 0.25 0.20 0.15 0.11 0.08 >draw : 0.37 0.41 0.44 0.47 0.49 0.50 0.49 0.47 0.44 0.41 >loss : 0.06 0.08 0.11 0.15 0.20 0.25 0.31 0.38 0.44 0.51 > > > >(Draw probability for equal opponents: 30%) >----------------------------------------------------------------------------- > 200 160 120 80 40 0 -40 -80 -120 <-delta >games: ---------------------------------------------------------------------- > 5: 11.62 7.34 4.29 2.31 1.15 0.53 0.22 0.09 0.03 0.01 > 10: #1.35# 0.54 0.18 0.05 0.01 0.00 0.00 0.00 0.00 0.00 > 15: 0.16 0.04 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 > 20: 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 > 25: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 > 30: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 > 35: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 > 40: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 > 45: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 > 50: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 >probability of: ------------------------------------------------------------- >win : 0.65 0.59 0.53 0.47 0.41 0.35 0.29 0.24 0.20 0.16 >draw : 0.22 0.24 0.27 0.28 0.30 0.30 0.30 0.28 0.27 0.24 >loss : 0.13 0.16 0.20 0.24 0.29 0.35 0.41 0.47 0.53 0.59 > > > # # entry: if I am 200 Elo better I still have only 1.35% chance of beating my >opp 10-0 > >========================== > >-Andrew-
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.