Author: Dieter Buerssner
Date: 01:56:23 01/05/04
Go up one level in this thread
On January 04, 2004 at 20:55:33, Peter Fendrich wrote: >On January 04, 2004 at 17:44:08, Dieter Buerssner wrote: > I really >>wonder, how exactly Elostat calculates the +/- margins to the calculated rating. >>Does anybody know an URL, where one can read this? > >I don't know how Elostat is working but it should be close to: > > s=SQRT(W(1-m)^2 + D(0.5-m)^2 + L(0-m)^2/(n-1)) Some parentheses are missing here? s=SQRT((W(1-m)^2 + D(0.5-m)^2 + L(0-m)^2)/(n-1)) > A=1.96 * s/SQRT(n) > > where s is estimated standard deviation > n is number of games > m is score/n > W,D,L is number of Wins, Draws and Losses respectivelly > A is the margin of error (for score, not rating points) > 1.96 is fethed from the Normal Distribution table to get > 95% reliability > SQRT is the square root > > Now we have an 95% interval of scores from m-A to m+A > > Compute the ratings for m-A and m+A and that's it. I tried with all different alternatives - your original formula, the one with parentheses added and also with W^2. In either case, the calculated number does not seem to make sense. For example (the case where one engine really is stronger with good confidence, despite of the close result): W=20 D=980 L=0 n=1000 m=0.51 With: CALC 24> s=sqrt((W*(1-m)^2 + D*(0.5-m)^2 + L*(0-m)^2)/(n-1)) 7.0035 A=4.34e-3 With ED = -400*lg(1/m - 1); lg() is logarithm to base 10 I calculate for m: 6.95; m+A: 9.97; m-A: 3.93 Elostat shows different numbers (besides the +7 rating difference): 68%: (+11,-0) 95%: (+22,-0) 99.7%: (+32,-1) In either case, one can conclude from the numbers, that the player with the wins is better. The error margin of your formula (+/-3 points) translates to a performance of 0.504 and 0.514 (instead of 0.51). Without losses, that would mean 8 or 28 wins instead of the 20. Seems reasonable, although the -12 compared to the +8 wins look a bit strange. I hope, I calculated correctly ... Now the other extreme, again 1000 games: W=510, D=0, L=490. s=5e-1, A=3.1e-2 ED for m+A: 28.55; m-A: -14.60 (for m again 7, of course) Looks again reasonable: Now we cannot be really confident anymore, that the winner is better. Elostat shows: 68%: (+11,-11) 95%: (+22,-22) 99.7%: (+32,-34) This time the margins are the same, as with your formula. Regards, Dieter
This page took 0.04 seconds to execute
Last modified: Thu, 07 Jul 11 08:48:38 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.