Author: Heiko Mikala
Date: 15:20:22 09/02/99
Go up one level in this thread
On September 02, 1999 at 15:52:17, Dave Gomboc wrote:
>I don't think that you addressed my point, namely:
>
> Less games are required to conclude with a certain confidence
> that one program is better than another when the results are
> lopsided than when they are not.
Oh yes, I think I did exactly address your point.
Look at these two match-fragments:
1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
A ½ 0 ½ 1 ½ 1 0 1 ½ 1 1 1 1 1 1 11.0/15
B ½ 1 ½ 0 ½ 0 1 0 ½ 0 0 0 0 0 0 4.0/15
1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
A 1 1 ½ 1 0 0 1 0 0 1 1 1 1 1 1 10.5/15
B 0 0 ½ 0 1 1 0 1 1 0 0 0 0 0 0 4.5/15
They both look very similar, don't they? So, if I understand you correctly, you
would conclude from both matches, that engine A will most definitely be stronger
than engine B, because the results are *very* "lopsided".
The first match-fragment from above are the first 15 games of my match between
CSTal II - Hiarcs.
The second match-fragment is from the second match that I quoted in my last
post. Go back and look at the endresult of the second match, which all in all
consisted of 73 games.
My point is: you can *not* conclude from a small lopsided match result, that one
engine is stronger than the other. No matter, how "lopsided" the result is, you
will need *much* more than only 15 or so games.
Greetings,
Heiko.
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.