Author: Walter Koroljow
Date: 14:12:10 02/03/01
Go up one level in this thread
On February 03, 2001 at 12:16:20, Uri Blass wrote: >On February 03, 2001 at 11:09:02, Walter Koroljow wrote: > <snip> I was glad to see your reply. We agree on many things. >>Let We = win expectancy of player A. Then the simplest hypotheses >>to use are: >> >>H0: We >= 0.5 >>H1: We < 0.5. >> >>If we reject H0, we necessarily accept H1. B is the better player at the >>confidence level of our test. > >I know statistics but the probability of 97% (when there is a 97% confidence) is >the probability to accept H0 when We=0.5 > >3% is the probability for error when We=0.5. >If We>0.5 the probability for error is even smaller so 3% is an upper bound for >the probability of an error but it is an upper bound when H0 is correct and this >model does not happen because often H1 is correct. Absolutely right, except for one thing -- when H1 is correct, it is correct to reject H0 (with H0 and H1 defined as they are above). In fact, if we knew that H1 occurred 50% of the time, we could say the error rate bound on false acceptance of H1 (false rejection of H0) was 1.5%, since H1 is automatically true half the time. > >I think the interesting question is what is the probability that H1 is correct >after knowing the result and the level of confidence does not give an answer for >it. Yes, this is true. But a bound is a useful result. I think Amir's calculation is arithmetically correct and very useful. He will not lose much money doing such calculations! > >You can say that you do not do the error of rejecting H0 when H0 is correct in >at least 97% of the cases. True, but I would say (see the argument above) that we do not need the condition that H0 is correct to make that statement. > >If you reject H0 often then it means that you are right in rejecting H0 in most >of the cases that you decide to reject H0. > >For example if you reject H0 in 50% of the cases that you test hypothesis then >you can say that the probability that H0 is wrong when you reject H0 is at least >94%. > >If you do not reject H0 often and Reject it only in 3% of the cases then it is >possible that you always make the wrong decision when you decide that H0 is >right when always We=0.5 I could not understand the last two paragraphs. How can it be a wrong decision to accept H0 when We = 0.5? I suspect there is a language problem here. > >> >>If we choose, as hypotheses: >> >>H0: We = 0.5 >>H1: We (not =) 0.5, >> >>then rejecting H0 does show the players are unequal, but does not say who the >>better player is. >> >>By the way, your argument using Bayes' theorem in another post is quite right, >>but not useful for computation as I am sure you know. > >I think that it can be useful for computation but the computation is more >complicated. > > One has to assume a value >>for win probability to begin with. > >It is possible to assume an aprior distribution for win probability to begin >with and this assumption is more logical. > >Unfortunately this is not the way that I learned to test hypothesis in >university because people prefer to do the things more simple and not to answer >the interesting questions. Assuming an a priori distribution and either arguing for its validity or doing a parametric study to see how the results vary with distribution would be very interesting. But that would satisfy almost none of the people who just want to know whether 10-0 or 60-40 is more significant. Can you imagine trying to justify a distribution assumption in this forum? Computing bounds by means of confidences does provide a reasonable answer to the question. I passionately agree that most interesting questions are usually not addressed. It is strange that that was the way in school. By the way, at work, our best-working military algorithms are Bayesian and they assume a priori distributions... It usually takes me a while to reply. Walter > >Uri >> >>Cheers, >> >>Walter
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.