Author: Uri Blass
Date: 09:16:20 02/03/01
Go up one level in this thread
On February 03, 2001 at 11:09:02, Walter Koroljow wrote: >On February 01, 2001 at 23:33:10, Uri Blass wrote: > >>On February 01, 2001 at 22:58:59, Uri Blass wrote: >><snipped> >>>From your previous post: >>> >>>"you assume the null hypothesis, which is that the >>>result is NOT significant and is a random occurrence between equals." >>> >>>You cannot calculate the probability that the winner is the better player by >>>assuming a model that does not exist. >> >>To make it clear you can calculate a probabilty for something that does not >>happen and to alculate 1- in order to calculate the probability that it happens >>but you assume the same space of events. >> >>When you say "I assume the null hypothesis" you change the space of events. >> >>The thing that is called the level of confidence is not something that has no >>importance because if the level of confidence is 99% then the probability >>(before knowing the result) of doing a mistake by deciding that the winner is >>better is at most 1%. >> >>It means that you get a wrong result in at most 1% pf the cases but it does not >>mean that you get right result in 99% of the cases and it is possible that you >>get no result in 97% of the cases,get a right result in 2% of the cases and get >>a wrong result in 1% of the cases and it means that you are correct only in 2/3 >>of the cases that you make a decision. >> >>Uri > >On February 01, 2001 at 23:33:10, Uri Blass wrote: > >>On February 01, 2001 at 22:58:59, Uri Blass wrote: >><snipped> >>>From your previous post: >>> >>>"you assume the null hypothesis, which is that the >>>result is NOT significant and is a random occurrence between equals." >>> >>>You cannot calculate the probability that the winner is the better player by >>>assuming a model that does not exist. >> >>To make it clear you can calculate a probabilty for something that does not >>happen and to alculate 1- in order to calculate the probability that it happens >>but you assume the same space of events. >> >>When you say "I assume the null hypothesis" you change the space of events. >> >>The thing that is called the level of confidence is not something that has no >>importance because if the level of confidence is 99% then the probability >>(before knowing the result) of doing a mistake by deciding that the winner is >>better is at most 1%. >> >>It means that you get a wrong result in at most 1% pf the cases but it does not >>mean that you get right result in 99% of the cases and it is possible that you >>get no result in 97% of the cases,get a right result in 2% of the cases and get >>a wrong result in 1% of the cases and it means that you are correct only in 2/3 >>of the cases that you make a decision. >> >>Uri > >Your objection is eliminated if the right hypotheses are used and correctly >interpreted. Let We = win expectancy of player A. Then the simplest hypotheses >to use are: > >H0: We >= 0.5 >H1: We < 0.5. > >If we reject H0, we necessarily accept H1. B is the better player at the >confidence level of our test. I know statistics but the probability of 97% (when there is a 97% confidence) is the probability to accept H0 when We=0.5 3% is the probability for error when We=0.5. If We>0.5 the probability for error is even smaller so 3% is an upper bound for the probability of an error but it is an upper bound when H0 is correct and this model does not happen because often H1 is correct. I think the interesting question is what is the probability that H1 is correct after knowing the result and the level of confidence does not give an answer for it. You can say that you do not do the error of rejecting H0 when H0 is correct in at least 97% of the cases. If you reject H0 often then it means that you are right in rejecting H0 in most of the cases that you decide to reject H0. For example if you reject H0 in 50% of the cases that you test hypothesis then you can say that the probability that H0 is wrong when you reject H0 is at least 94%. If you do not reject H0 often and Reject it only in 3% of the cases then it is possible that you always make the wrong decision when you decide that H0 is right when always We=0.5 > >If we choose, as hypotheses: > >H0: We = 0.5 >H1: We (not =) 0.5, > >then rejecting H0 does show the players are unequal, but does not say who the >better player is. > >By the way, your argument using Bayes' theorem in another post is quite right, >but not useful for computation as I am sure you know. I think that it can be useful for computation but the computation is more complicated. One has to assume a value >for win probability to begin with. It is possible to assume an aprior distribution for win probability to begin with and this assumption is more logical. Unfortunately this is not the way that I learned to test hypothesis in university because people prefer to do the things more simple and not to answer the interesting questions. Uri > >Cheers, > >Walter
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.