Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: A question about statistics...

Author: Dieter Buerssner

Date: 01:56:23 01/05/04

Go up one level in this thread


On January 04, 2004 at 20:55:33, Peter Fendrich wrote:

>On January 04, 2004 at 17:44:08, Dieter Buerssner wrote:
>
 I really
>>wonder, how exactly Elostat calculates the +/- margins to the calculated rating.
>>Does anybody know an URL, where one can read this?
>
>I don't know how Elostat is working but it should be close to:
>
>      s=SQRT(W(1-m)^2 + D(0.5-m)^2 + L(0-m)^2/(n-1))

Some parentheses are missing here?

s=SQRT((W(1-m)^2 + D(0.5-m)^2 + L(0-m)^2)/(n-1))


>      A=1.96 * s/SQRT(n)
>
>      where s is estimated standard deviation
>            n     is number of games
>            m     is score/n
>            W,D,L is number of Wins, Draws and Losses respectivelly
>            A     is the margin of error (for score, not rating points)
>            1.96  is fethed from the Normal Distribution table to get
>                  95% reliability
>            SQRT  is the square root
>
>   Now we have an 95% interval of scores from m-A to m+A
>
>   Compute the ratings for m-A and m+A and that's it.

I tried with all different alternatives - your original formula, the one with
parentheses added and also with W^2. In either case, the calculated number does
not seem to make sense. For example (the case where one engine really is
stronger with good confidence, despite of the close result):

W=20
D=980
L=0
n=1000
m=0.51

With:

CALC 24> s=sqrt((W*(1-m)^2 + D*(0.5-m)^2 + L*(0-m)^2)/(n-1))
 7.0035
A=4.34e-3

With

 ED = -400*lg(1/m - 1); lg() is logarithm to base 10

I calculate for m: 6.95; m+A: 9.97; m-A: 3.93

Elostat shows different numbers (besides the +7 rating difference):

68%:   (+11,-0)
95%:   (+22,-0)
99.7%: (+32,-1)

In either case, one can conclude from the numbers, that the player with the wins
is better. The error margin of your formula (+/-3 points) translates to a
performance of 0.504 and 0.514 (instead of 0.51). Without losses, that would
mean 8 or 28 wins instead of the 20. Seems reasonable, although the -12 compared
to the +8 wins look a bit strange. I hope, I calculated correctly ...

Now the other extreme, again 1000 games: W=510, D=0, L=490. s=5e-1, A=3.1e-2
ED for m+A: 28.55; m-A: -14.60 (for m again 7, of course)

Looks again reasonable: Now we cannot be really confident anymore, that the
winner is better.

Elostat shows:

68%:   (+11,-11)
95%:   (+22,-22)
99.7%: (+32,-34)

This time the margins are the same, as with your formula.

Regards,
Dieter



This page took 0.05 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.