Computer Chess Club Archives




Subject: Re: Proving something is better

Author: Peter Fendrich

Date: 06:41:17 12/27/02

Go up one level in this thread

Finally I got some time to elaborate a bit about this.

(in the previous post I meant the bell curve surrounding the result is shrinking
and not surrounding 0.5)

You have elegantly showed that "Draws doesn't count" and that leads me to the
conclusion that something is wrong with our main question/defintiions.
What does "A is better than B" really mean?

Let's start with a small illustration.
We have a match result 6-4 (6 won and 4 loss) and get the prob 0.726 that "A is
better than B".

We have the same result but in addition also 1 million draws and get the same
probability that "A is better than B" by removing draws.

Now let's ask, still with exactly the same probabilities, "what is the
probability that A will win against B in a 10 game match". We will find that the
information from the first match would give us a good probability well beyond
50% that A will win.
By using the second result instead, with 1 million draws, we wouldn't give A
much more than 50% chance to win that match. The probability of a draw is for
each game, very close to 1.0.

This illustration shows that therre is information in a draw that we can't use
because of how the question is formulated or maybe because how "better" is

Why is it so
There are two different explanations.
1) Game results are discrete and in a 10 game match this is very obvious (see
the example) and we never play infinite matches.

2) In "normal" speak the word 'better' means that A is generally better than B
in the whole population. Not just between the two of them. "A will probably do
better than B against all players in total". The match result between A and B
has to show a certain degree og confidence before we can make any statement
about the whole population. In short, If A beats B with 10-0 there is some
correlation with A being better than B against the rest of the population.
Letting "better" mean any figure above 0.5 is not useful in this case. It's like
telling the time for the next TV program with 10 decimals. TV programs doesn't
start that exact even if we can express and compute it.

What to do
I have a few suggestions that I would like to discuss:

1) Better utilisation of computer time. If I have time for 20 games it's better
to select 10 players and let A and B meat them respectively.
The meaning of better will be better.

2) Use some degree of better, for instance 60% (instead of 50%) as the lower
limit. "A beats B with at least 60%" with a probability of x%. It's hard to tell
anything about probability against the rest of the population but maybe some a
priori distribution can be used.

In both cases draws has to be counted because they are part of the question.


This page took 0.24 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.