Author: Christophe Theron
Date: 08:03:38 03/09/02
Go up one level in this thread
On March 09, 2002 at 02:46:42, TEERAPONG TOVIRAT wrote:
>
>
>I see the correlation between score ratio and the number of games
>at specific % of confident interval.
>Would u please calculate this correlation for me?
>
>score the number of games at 90% confident interval
>60% ?
>70% ?
>80% ?
>90% ?
>
>Thanks in advance,
>Teerapong
I personally use the following tables. Study them and you will quickly
understand that the number of games needed to draw a reasonable conclusion
exceeds what common sense believes. Common sense sucks on this matter, don't
trust your feelings.
Explanations:
1) I know that assuming 1/3 chances for wins, draws and losses is not correct,
but I think it's close enough to reality and does not invalidate the reliability
of these tables.
2) How to read the tables: for example, if you want 90% reliability in your
conclusions and have played 10 games, then you must assume a +/-20% error margin
in the winning percentage of the winner (which translate to a +/-140 elo margin
of error). So if program A beats program B by 65% in a 10 games match, then you
cannot even tell which program is better. Play more games.
3) These tables should be taken with a statistical grain of salt. So if you
don't understand the concept of margin of error, reliability percentage of a
result and so on, just forget about them and go back to tic-tac-toe. ;)
Reliability of chess matches
(assuming each opponent has 1/3 chances to win, 1/3 to loose and 1/3 to draw)
90% confidence
Games %err+/- elo+/-
10 20 140pts
20 15 105pts
25 14 98pts
30 12 63pts
40 10 70pts
50 9 56pts
100 6.5 35pts
200 4.72 33pts
400 3.34 23pts
600 2.66 19pts
800 2.39 17pts
1000 2.12 15pts
1200 2.00 14pts
1400 1.81 13pts
1600 1.66 12pts
80% confidence
Games %err+/- elo+/-
10 15 105pts
20 11 77pts
25 10 70pts
30 9 63pts
40 8 56pts
50 7 49pts
100 5.0 35pts
200 3.75 26pts
400 2.60 18pts
600 2.15 15pts
800 1.86 13pts
1000 1.66 12pts
1200 1.46 10pts
1400 1.40 10pts
1600 1.34 9pts
70% confidence
Games %err+/- elo+/-
10 15 105pts
20 10 70pts
25 8 56pts
30 8 56pts
40 6.3 44pts
50 6.0 42pts
100 4.0 28pts
200 3.0 21pts
400 2.2 15pts
600 1.7 12pts
800 1.5 11pts
1000 1.3 9pts
1200 1.24 9pts
1400 1.14 8pts
1600 1.04 7pts
Christophe
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.