Author: Erik Bergren
Date: 07:54:49 08/06/03
Go up one level in this thread
> >Hi George. >I have a question. I am not a mathematician and I'm sure many who visit here >aren't either. Why is 600-700 games the magic number? Dann Corbit explained >that mathematics will draw the line as to how many games it would take to show >which program is stronger. If after 100 games one program is trounced could one >not with reasonable assurance say one program plays chess better than the other? > What would be the mathematical formula for wins and losses after 100 games to >say with reasonable assurance that program A is stronger than program B? What >if one program won 85 out of one hundred games with no only 5 draws? >Thanks . All "Probability Ratios" stay constant regardless of the number of samples (such as 100 or 600). This must be true simply because: time has no effect on probabilistic results. There would be an exception to that if: the programs were able to learn (but that effect would not show up until a huge number of games were played). As to your question of "assurance" that 85 wins out of 100 shows one to be superior to the other: Just model the problem with pennies: assume an equal chance to a flip resulting in a heads up landing of the penny. Thus getting 85 heads out of 100 would be quite unlikely indeed ( just have your computer calculate all possible out comes, and find the ratio of the total of all of those, to those that have more than or equal to 85 heads out of 100). You will thus be shown that one program not being better than the other (thus them being equal in strength), after 85 wins out of 100, is less likely than 1 in 1000000 ( I do not have a computer with me to find the exact number).
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.