Author: Kolss
Date: 05:32:34 10/20/05
Go up one level in this thread
On October 20, 2005 at 05:52:13, Kurt Utzinger wrote: >On October 20, 2005 at 05:20:31, Ray Banks wrote: > >>On October 20, 2005 at 03:53:50, Graham Banks wrote: >> >>>In EVERY match of the 32 played to date, the engine that was first to 10.5 (best >>>of 20 games) was also first to 15.5 (best of 30 games). >>> >>>If this trend continues, it will be good proof that 20 game matches are long >>>enough in most cases to show up the stronger of the two participants. >>> >>>Any comments? >> >>As mentioned below, proof only if there is a big differential between the >>engines. If the engines are relatively close, then 20 or even 30 is not enough. > > Agreed: a more reliable number of sufficient games > in a match bewtween two engines is 50. > Regards > Kurt Yes, and better yet is a match of 80 games. And even better is 138 games. And... Sorry, couldn't resist :-). The point has already been made though. 20 games may be sufficient if the difference in playing strength between two programs / versions is very large. A score of 18.5 : 1.5 is convincing enough (i.e. you can say with high statistical certainty that the former program really is better than the other one). The closer programs are in strength, the more games are required to resolve the difference. A 51.5 : 48.5 does not tell you in any significant way which program is stronger. If you want to find / prove a 5 Elo difference, 1000 games is *NOT* enough. Simple as that... Best regards - Munjong.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.