Author: Uri Blass
Date: 08:09:05 04/14/05
Go up one level in this thread
On April 14, 2005 at 09:37:06, pavel wrote: >Not yet. > >In discussion with Dann. I wanted to try out to see how many games it would take >to achieve 1 ELO error margin. While I do agree with Dann that it would take >100,000s games to achieve this. I would like to see this in practice. > >So after the first 1000 games: >------------------------------------------- > >4/14/2005 9:27:02 AM : > > Program Elo + - Games Score Av.Op. Draws > > 1 Aristarch 4.50 : 2519 19 19 1000 55.4 % 2481 23.3 % > 2 Yace Paderborn : 2481 19 19 1000 44.6 % 2519 23.3 % > >------------------------------------------- >Individual statistics: > >1 Aristarch 4.50 : 2519 1000 (+437,=233,-330), 55.4 % > >Yace Paderborn : 1000 (+437,=233,-330), 55.4 % > >2 Yace Paderborn : 2481 1000 (+330,=233,-437), 44.6 % > >Aristarch 4.50 : 1000 (+330,=233,-437), 44.6 % > >------------------------------------------- >Games : 1000 (finished) > >White Wins : 407 (40.7 %) >Black Wins : 360 (36.0 %) >Draws : 233 (23.3 %) >Unfinished : 0 > >White Perf. : 52.4 % >Black Perf. : 47.6 % > >ECO A = 321 Games (32.1 %) >ECO B = 220 Games (22.0 %) >ECO C = 105 Games (10.5 %) >ECO D = 196 Games (19.6 %) >ECO E = 158 Games (15.8 %) >------------------------------------------- > >Ponder = OFF >TB = ON >Time Control = 2 minutes a side (I could use 1 min a side and save a lot of > time, but risking too many games resulting in > flags) >Book = Own Book >Learn = On >GUI = Chessbase >Mashine = Pentium IV 2.66 >Hash = 50mb > > >I will try to update every thousand games or so. > >Cheers :) >pavs I think that all this experiment will say nothing about real games in tournaments because of learning(nobody except you test by playing a match of 100,000 games between programs). The error margins are based on the assumptions that the game are independent event. When you play matches with learning the games are dependent. If you want to get small error margins the only way is to play nunn match with 50,000 different positions and no learning. Of course in that case you will not get the playing strength of the programs but you need to decide what you prefer and if you want to get small error margins you need to give up finding the playing strength of the programs. I think that if you play a match of 100,000 games with learning you may get results like 90,000:10,000 because after 10,000 games one program will beat the second program in almost every game thanks to learning(espacially in the case that the second program has only learning that is based on book and not positional learning). Movei may lose 90,000:10,000 or worse against program at the same level in tournaments only because Movei has no positional learning. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.