Author: Vasik Rajlich
Date: 06:44:13 10/16/04
Go up one level in this thread
On October 16, 2004 at 04:18:06, martin fierz wrote:
>hi graham!
>
>in the last days you suggested that junior seriously underperformed in bilbao
>and even wrote a small program to prove your point. you were quite undeterred by
>all the people saying "too little games" because you were looking at the results
>your simulator gave you. i'd like to explain why your argument is flawed, and i
>will use your little program to do it :-)
>
>let's see, i will take the probabilities to win, lose and draw for the average
>computer player to be 50%, 40% and 10% (that is my 'best estimate' based on the
>actual results).
>
>what do i get:
>DJ won 0 points in 0.02% of the tournaments
>DJ won 0.5 points in 0.18% of the tournaments
>DJ won 1 points in 1.16% of the tournaments
>DJ won 1.5 points in 5.23% of the tournaments
>DJ won 2 points in 14.09% of the tournaments
>DJ won 2.5 points in 24.73% of the tournaments
>DJ won 3 points in 29.12% of the tournaments
>DJ won 3.5 points in 19.21% of the tournaments
>DJ won 4 points in 6.26% of the tournaments
>
>now, the disagreement begins as to what these numbers mean. you are implying
>that the above numbers indicate that DJ has a very, very low probability of
>scoring only 1.5 points. that in itself is quite true, but *every* single result
>is rather unlikely. what you really need to do is compare the most likely
>outcome (scoring 3 points) against the actual outcome (if you believe that the
>underlying winning probabilities are the truth). and NOT compare every single
>result vs 100%!
>
>so: most probable outcome would be all computers score 3 points, with a joint
>probability of this happening being (0.2912)^3 = 0.0247 = 2.5%
>the actual outcome had a probability of (0.1921)^2*(0.0523) = 0.0019 = 0.2%.
>
>these numbers show: the probability of any SINGLE result is very low - even the
>most probable result only happens in 2.5% of all cases. the probability of the
>actual result happening is 13 times smaller. in this sense, if you want to stick
>to your hypothesis that all computers were of similar strength, then this was a
>slightly unusual result. but it was most definitely NOT an improbable result.
>your mistake seems to be that you take the probability of a result occurring,
>and compare it to 1 ("0.2% is very unlikely - 1 in 500"). instead, you have to
>compare it with the probabilty of the most likely result occurring, and then
>things don't look improbable at all (0.2% vs 2.5% - 1 in 13). did i make this
>point clear enough?
>
>
>now, with all this said and done, the result gets even more likely if you factor
>in the playing strength of the humans. the match was very weird in the sense
>that they had 4 rounds for 3 players each, so one program had to play one human
>twice. bad luck for junior, it had to play topalov twice. he was the
>highest-rated human of the lot, and he just came back from a stunning
>performance at the fide world chess championship. david levy writes
>(http://www.chessbase.com/newsdetail.asp?newsid=1956)
>
>"But whatever the level of preparation of team GM it did not show itself to good
>effect in most of the games, although Topalov appeared to have a much better
>understanding of how computers play chess than did either of his team-mates."
>
>so topalov was the highest-rated + best prepared for this competition according
>to levy (and he knows a bit something about both chess and computer chess). if i
>take the 1-in-13 chance of the actual result happening, and add that topalov was
>the strongest player on the human side, that will make the actual result more
>probable of course, at least 1-in-10 i would guess compared to a "most likely"
>result. now i don't call that unlikely. do you?
>
>cheers
> martin
You're right that the argument that "the chance of this result is only 5.23%" is
bogus. The right form of that argument is: the chance of this result (1.5/4) or
less is: 0.02% + 0.18% + 1.16% + 5.23%. That's still a pretty low number.
In addition, the chance of Fritz and Hydra scoring 7/8 (or more) given your
estimate of 50%, 40% and 10% would be an even lower number. (In fact I think
your numbers are too optimistic for the humans - unfortunately.)
If you consider Junior to be able to stand humans as well as Fritz and Hydra,
what happened was an extremely anomalous result.
Vas
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.