Author: Peter Fendrich
Date: 15:25:53 01/23/04
Go up one level in this thread
On January 23, 2004 at 15:24:31, Dann Corbit wrote: Yes, this applies to the error bars and the Elo ratings. Not the results as such. After only playing for instance 10 games will the error bars not only be very wide but also very unreliable due to the bad adaptation of the Trinomial distribution to the Normal distribution. 30 games is probabably good enough to eliminate the error of inexact adaption to the Normal distribution. There are other "experiment setup" errors that I suspect biases the result even more. Things like too few opponents or non optimal setups of the engines and sofort. /Peter >30 experiments is a fairly standard rule as to when you should start to trust >the results for experimental data. > >From: >http://www.twoplustwo.com/mmessay8.html >"A good rule of thumb is to have at least 30 observations (playing sessions) for >the estimate to be reasonably accurate. However, the more the better, unless for >some reason you think the game for which you are trying to estimate your >standard deviation has changed significantly over some particular period of >time." > >From: >http://www.odu.edu/sci/xu/chapter3.pdf >"C. The Reliability of s as a Measure of Precision - the more measurements that >are made, the more reliable the value obtained for s. Usually 20 - 30 >measurements are necessary." > >From >http://www.stat.psu.edu/~resources/ClassNotes/ljs_21/ljs_21.PPT#11 >Concerning the central limit theorem, we have this: >Even if data are not normally distributed, as long as you take “large enough” >samples, the sample averages will at least be approximately normally >distributed. >Mean of sample averages is still mu >Standard error of sample averages is still sigma/sqrt(n). >In general, “large enough” means more than 30 measurements. > > >Of course, the more the merrier, when it comes to measurements.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.