Author: martin fierz
Date: 01:18:06 10/16/04
hi graham!
in the last days you suggested that junior seriously underperformed in bilbao
and even wrote a small program to prove your point. you were quite undeterred by
all the people saying "too little games" because you were looking at the results
your simulator gave you. i'd like to explain why your argument is flawed, and i
will use your little program to do it :-)
let's see, i will take the probabilities to win, lose and draw for the average
computer player to be 50%, 40% and 10% (that is my 'best estimate' based on the
actual results).
what do i get:
DJ won 0 points in 0.02% of the tournaments
DJ won 0.5 points in 0.18% of the tournaments
DJ won 1 points in 1.16% of the tournaments
DJ won 1.5 points in 5.23% of the tournaments
DJ won 2 points in 14.09% of the tournaments
DJ won 2.5 points in 24.73% of the tournaments
DJ won 3 points in 29.12% of the tournaments
DJ won 3.5 points in 19.21% of the tournaments
DJ won 4 points in 6.26% of the tournaments
now, the disagreement begins as to what these numbers mean. you are implying
that the above numbers indicate that DJ has a very, very low probability of
scoring only 1.5 points. that in itself is quite true, but *every* single result
is rather unlikely. what you really need to do is compare the most likely
outcome (scoring 3 points) against the actual outcome (if you believe that the
underlying winning probabilities are the truth). and NOT compare every single
result vs 100%!
so: most probable outcome would be all computers score 3 points, with a joint
probability of this happening being (0.2912)^3 = 0.0247 = 2.5%
the actual outcome had a probability of (0.1921)^2*(0.0523) = 0.0019 = 0.2%.
these numbers show: the probability of any SINGLE result is very low - even the
most probable result only happens in 2.5% of all cases. the probability of the
actual result happening is 13 times smaller. in this sense, if you want to stick
to your hypothesis that all computers were of similar strength, then this was a
slightly unusual result. but it was most definitely NOT an improbable result.
your mistake seems to be that you take the probability of a result occurring,
and compare it to 1 ("0.2% is very unlikely - 1 in 500"). instead, you have to
compare it with the probabilty of the most likely result occurring, and then
things don't look improbable at all (0.2% vs 2.5% - 1 in 13). did i make this
point clear enough?
now, with all this said and done, the result gets even more likely if you factor
in the playing strength of the humans. the match was very weird in the sense
that they had 4 rounds for 3 players each, so one program had to play one human
twice. bad luck for junior, it had to play topalov twice. he was the
highest-rated human of the lot, and he just came back from a stunning
performance at the fide world chess championship. david levy writes
(http://www.chessbase.com/newsdetail.asp?newsid=1956)
"But whatever the level of preparation of team GM it did not show itself to good
effect in most of the games, although Topalov appeared to have a much better
understanding of how computers play chess than did either of his team-mates."
so topalov was the highest-rated + best prepared for this competition according
to levy (and he knows a bit something about both chess and computer chess). if i
take the 1-in-13 chance of the actual result happening, and add that topalov was
the strongest player on the human side, that will make the actual result more
probable of course, at least 1-in-10 i would guess compared to a "most likely"
result. now i don't call that unlikely. do you?
cheers
martin
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.