Author: Uri Blass
Date: 07:37:50 12/26/00
Go up one level in this thread
On December 26, 2000 at 09:21:24, Amir Ban wrote: >On December 25, 2000 at 11:55:51, Uri Blass wrote: > >>On December 25, 2000 at 10:08:44, Amir Ban wrote: >> >>>On December 24, 2000 at 13:11:49, Christophe Theron wrote: >>> >>>>On December 24, 2000 at 09:09:27, Jeroen Noomen wrote: >>>> >>>>>My congratulations to Vishy Anand, for winning the >>>>>FIDE World Championship 2000! >>>>> >>>>>3,5-0,5 in the final against Shirov, that leaves no >>>>>discussion whatsoever. Anand was the best, remained >>>>>unbeaten and scored a clear victory in the final. >>>>>Well done! >>>>> >>>>>Jeroen >>>> >>>> >>>>I am still absolutely amazed that a World Championship can be decided this way. >>>> >>>>A score of 3.5-0.5 is not statistically significant, not even with a low >>>>confidence. >>>> >>>>It is now clear, at least amongst the experienced computers chess operators, >>>>that such a result means NOTHING. >>>> >>>>I think that the computer chess community is on some topics much more advanced >>>>than the human chess community. For example the human chess community has >>>>adopted the ELO rating system, but still ignores most of the basic rules of this >>>>system (margin of error, level of confidence). The computer chess community is >>>>aware of these rules, and you can find these parameters published in the SSDF >>>>rating list for example. >>>> >>>> >>>> >>>> Christophe >>> >>>Chess games are not random events. >>> >>>You failed to do the math: 3.5-0.5 *is* significant, with about 95% confidence. >> >>I do not know how do you get the 95% confidence. >> >>You should define the assumptions that you make in order to find if it is >>significant with 95% confidence. >> >>There is also a problem in deciding if the result is significant. >> >>If you do a small change in the program then 7-0 result can be not significant >>from your point of view. >> >>7-0 is very rare result between equal players but if you know that the change in >>the program is small before testing then the probability that the weaker side >>won 7-0 when you know that one side won 7-0 is not small enough. >> >>Let assume that the probability for the weaker side to win a game is 0.3 when >>the probability for the stronger side to win a game is 0.35 >> >>the probability of the weaker side to win 7 games when you know that one side >>won is 0.3^7/(0.3^7+0.35^7)=0.2536... and it means that there is a probability >>of more than 25% that you did a bad change inspite of the 7-0 result >> > >The probability is 0.3^7 = 0.0002 > >If you assume there are no draws (or don't count them) the probability is about >0.004 > >Amir I know it but the probability that the weaker side won 7-0 when you know that one side won 7-0 is 0.2536... and this is the important probability when you do a small change in your program. If you change your program by changing the value of the pawn and you want to test if the change is positive or negative by games you know that the difference cannot be very big. If you assume that the weaker side wins 30 % of the games and that the stronger side wins 35% of the games then 0.2536 is the probability that the new version is weaker after you know the result. The practical situation is even worse than it because most of the changes that you want to test are even smaller than 20 elo. 7-0 and 107-100 have the same confidence if you know that the stronger side is 20 elo better and you have to decide which side is stronger. Practically you do not know that the stronger side is 20 elo better and 20 elo is only an upper bound. This is the reason that 7-0 is more safe than 107-100 to decide which side is stronger without errors but inspite of it the rule to stop when there is a difference of 7 seems logical(except the fact that 7 may be too small). 107-100 is less safe than 7-0 but on the other side when you get 107-100 you have a basis to believe that the difference is smaller and when the difference is smaller it is less important to have big probability of being correct in your decision. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.