Author: Uri Blass
Date: 06:59:45 12/26/00
Go up one level in this thread
On December 26, 2000 at 02:30:18, Bruce Moreland wrote: >On December 25, 2000 at 23:41:38, Roger D Davis wrote: > >>Actually, statistical significance is a function of sample size and effect size. >>Sample size is number of persons in a study, for example, or the number of times >>a particular value in an experiment was recorded. The effect size is regarded as >>the true difference between the means of the samples on the relevant >>variable(s). Statistical significance is always a function of sample size AND >>effect size. If the effect size is large, as with a large difference of ability >>between two players, then fewer games will be needed to determine who is the >>significant winner (as opposed to just the winner, which may be statistically >>meaningless). >> >>Conversely, it is certainly possible that the match could stretch out forever >>between evenly matched opponents. Obviously, this statement is tautological, >>since that's what it means to be evenly matched in the first place...it is >>trivially true. One way of dealing with this possibility is to set a minimum and >>maximum number of games. If players reach the maximum number of games without a >>statistically significant result, then the tie could go to the old champion, or >>there could be no champion. >> >>Alternately, a draw could be agreed unless both players wanted to play on, in >>which the match could continue another ten games or whatever. >> >>There are many ways of constructing it that I would consider interesting. >>Obviously, Bruce has another opinion. > >I think it could stretch out forever between *unevenly* matched opponents. > >We currently have a gap of 75 Elo points between #1 and #2. The gap between #2 >and #3 is 2 points, then 18 more is #4. In fact, that 75 points can get you all >the way from #2 to #11, there are 75 points between these two guys, with 8 >players in between. > >But let's assume to start with that you get players this amazing 75 Elo points >apart. How many games would it take, on average, to get statistical >significance? > >I wrote a program that figures some of this out. I chose to look at the case of >50 games, since that's about twice as many as have been played in the past, and >I take this as an impractical number. I don't think you'd find participants who >would agree to a 50-game match. It would be even less likely if you told them >they would have to demonstrate clear superiority or they'd share the prize. > >Ok, let's look at 50 games. I believe my program shows, that with a decent draw >percentage (I chose 45%) It seems that there are more draws between humans. If you look at the match between kramnik and kasparov there were 13 draws out of 15. If you assume probability of 25% for the winner,5% for the loser and 70% draws then the standard deviation is smaller and you need less games to get significant result. Another point is that the games are not independent events. The first match between kasparov and karpov demonstrated it when karpov was leading 5-0 and decided to quit the match when he was leading 5-3 because he was tired. The reason that he decided to quit is that he knew that he was tired and that kasparov has bettter chances if the match continues. Karpov did not think about the games as independent events(otherwise he could prefer to continue and be almost sure of winning even if kasparov is slightly better). Even if you assume that the probability for kasparov to win a game is 0.2 and the probability of karpov to win a game is 0.1(I ignored white and black for doing the problem more simple) karpov had probability of 19/27 to win but karpov understood that the situation is worse than that and starting a new match from 0-0 result is better for him. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.