Author: Uri Blass
Date: 13:02:27 01/12/01
Go up one level in this thread
On January 12, 2001 at 15:34:30, Dann Corbit wrote: >On January 12, 2001 at 14:47:21, Uri Blass wrote: > >>On January 12, 2001 at 14:27:08, Dann Corbit wrote: >> >>>If the math says that it >>>is uncertain, then it is uncertain. To then claim that it is certain is >>>completly wrong. >> >>You will always be uncertain. >>It is always possible with probability that is more than 0 that a program that >>played random moves will lead the ssdf list. >> >>If you are sure in 99% you are also uncertain. > >If the uncertainty is enormous (e.g. x-bar might be as low as 2350 within one >standard deviation) then it is not proven. One standard deviation is a minimal >level to consider partly proven. It is still highly uncertain. But given two >standard deviations, it is 97% certain. > >To accept a certainty that is well under 50% and accept something as proven is >bad. If your life was on the line, you would not accept that unless you had no >alternatives. > >>The probability of confidence is based on some assumptions when the assumptions >>are wrong. >> >>Some examples for wrong assumptions that the ssdf use: >> >>1)The result of games are independent events(false, programs learn). >>2)The probability for white to win is the same as the probability of black to >>win(again false). > >The ELO model makes no such assumptions. Therefore, such assumptions must be >poor statements in the description by the SSDF. I think that these assumptions are based on the elo model. I did not read them as statement by the ssdf. I remember that the elo model makes an assumption that the ability of players is distibuted noramlly with standard deviation 200. I think that it assumes a simple model with no draws in order to calculate the expected result between 2 players. It means that the expected result is based only on the rating of the players and previous games are not relevant. > >The number of games played as white and black against a common opponent are the >same. Therefore, that particular difference is irrelevant. It still make difference because the varience of the result is smaller if we assume difference probability for white and black. I will give an extreme case to demonstrate it. Suppose that 2 players are equal: Case 1:The probability to win is 50% and the colour is not relevant. Case 2: the probability of white to win is 100%. In cases 1 the result is based on the binomical distribution and in case 2 the result is always draw assuming the same number of games with white and black. It means that in case 2 the varience of the result is smaller(in this example 0). > >Learning is a separate issue, and is (indeed) a flaw in the model. However, the >programs that learn really do become better. And they probably win more games. >Hence the results are not innaccurate. The results are dependent in the number of games and the programs that learn better will become better with more games. It means that the decision to play 40 games between 2 programs and not 80 games change the rating. <snipped> >I don't have any problem with trusting your feelings. But do you believe that >these feelings prove an assertion? That is where I think errors are found. I think that using the level of confidence of the ssdf is also trusting some feelings that the errors in the model are small enough to do the level of confidence correct. > >>Without doing it I can only say that I know nothing and even do not know the >>level of confidence. > >I am not sure what you mean by this statement. Can you clarify it? I mean that I can say that I do not know the level of confidence that one program is better than another one from mathematical point of view and I prefer to trust my guess about the question which program is better. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.