Author: Peter Fendrich
Date: 11:23:28 08/14/98
Go up one level in this thread
On August 14, 1998 at 11:36:56, Dan Homan wrote: >On August 14, 1998 at 10:42:04, Bruce Moreland wrote: > >> >>On August 14, 1998 at 06:08:36, Dan Homan wrote: >> >>>On August 12, 1998 at 09:15:19, Bruce Moreland wrote: >>> >>>> >>>>On August 11, 1998 at 06:52:02, Tony Hedlund wrote: >>>> >>>>> >>>>>>So I think 4-0 actually turns out to be a significant result. If you score 4-0, >>>>>>you can say that there is a very good chance that the one with the wins is >>>>>>better than the ones with the losses. >>>>>> >>>>>>You can't say this if you pick out a string of 4 wins in a row in the midst of a >>>>>>longer match, since you might be selecting a fluke case, but if you just start >>>>>>from scratch, and get 4-0, you should be able to stop. In fact I think you >>>>>>might be able to stop if you get 3.5 - 0.5, but I am less certain of this case. >>>>>>Someone who has more statistics than I may be willing to comment on this. >>>>> >>>>>Recently I played the match Shredder2 P200 MMX 64MB - Rebel8 P90 16MB. >>>>>Rebel won the first four games but Shredder won the match with 11-9. >>>> >>>>That shouldn't happen very often. >>>> >>>>bruce >> >>If this does happen more often than it should, perhaps some effort could be >>expended to figure out why. I don't know why it would happen more often than it >>should. >> >>>The problem with small number statistics is that they can be very >>>mis-leading. A 4-0 result in a 4 game match between nearly equal >>>programs (with 20% draw chances) happens about 1/40 th of the time. >>>A 3.5-0.5 (or better) result happens about 1/13 th of a time. >> >>If you want to be able to say, "program A is stronger than program B, by at >>least a little bit", would you rather have a 4-0 result or a 105-95 result? >> >>You should get a bogus result from 4-0 only 2.5% of the time by your >>calculation, > >Actually I think my calculation was wrong. I didn't consider the chance >of the weaker program going 4-0.... see below. > >>how much do you want to bet that you'd get a bogus result from >>105-95 a lot more of the time? > >Of course, but my point is that if you think you are learning anything >useful from 4-0 result in a 4 game match between top programs you are >wrong. I initially thought this was a great idea until I thought about >it - the more I think about it the worse it gets. I don't understand how you guys get these results. In my world probability is completely different from confidence. If the probability is odds, chance or whatever, the confidence is a meassurement of how sure we can be of the probability value itself. The information given by a 200-game match gives far more confident probabilities than a 4-game match. In the 4-game match it's not even applicable to use the term confidence. It's as worthless to compute like meassuring 1/100 of a seconds with you wrist-watch. //Peter
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.