Author: Dieter Buerssner
Date: 10:04:55 04/14/05
Go up one level in this thread
On April 14, 2005 at 11:17:56, Kolss wrote: >On April 14, 2005 at 09:37:06, pavel wrote: > >>Not yet. >> >>In discussion with Dann. I wanted to try out to see how many games it would take >>to achieve 1 ELO error margin. While I do agree with Dann that it would take >>100,000s games to achieve this. I would like to see this in practice. >> >>So after the first 1000 games: >>------------------------------------------- >> >>4/14/2005 9:27:02 AM : >> >> Program Elo + - Games Score Av.Op. Draws >> >> 1 Aristarch 4.50 : 2519 19 19 1000 55.4 % 2481 23.3 % >> 2 Yace Paderborn : 2481 19 19 1000 44.6 % 2519 23.3 % >The standard error (SE) goes down proportional to the square root of the number >of events, i.e. games in this case - provided, as Uri pointed out, that the >games are independent samples. Assuming that the fractions of wins, draws, and >losses stay the same over the next games, you can expect to get the SE to a >tenth (1.9) if you increase the number of games by a factor of 100 (100000). >Your data would suggest that you need about 400000 games to get the SE down to 1 >ELO point. Have fun! Actually, the standard deviation (which you call Standard error - yes?) is one half of what Elostat shows. So the 100000 games will be good for 1%. It depends on the strength difference, too. In this match, both opponents were of similar strength, in which case one needs the fewest games, to get down the error. When one engine score 95%, you will need many more games. This also is in accordance to what one would expect. It is difficult to judge, but I would fear, that with 100000 games, the games are not really independent, even when both players use a very wide opening book. IIRC, the typical "grandmaster only PGNs" from which many amateur opening books are created, have a few 10000 games. Even if they all were different in the first 20 moves or so, typical book selection algorithms of amateur engines would not use them all, or would use the more popular systems more often than lesser played variants. So in practice, it just will be impossible - even with lots of patience and very fast time controls. Regards, Dieter
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.