Author: Joseph Ciarrochi
Date: 19:45:36 12/21/05
I was glad to see that the author of rybka plans to make Rybka into a way cool
analysis tool, for those of us who are using computers to improve our own play .
A key place that we need help is in choosing between different opening
variations (especially after move 10 or so and theory is often not developed).
Unfortunately, computers are not so good at this, as Dann Corbit and others have
pointed out.
Below is an example of this. Most engines judge this position as moderately to
maybe strongly favouring white. Yet when I played engines against themselves in
this opening (thus controling for skill), white only wins 40% (see example
below)
i would like to make a humble suggestion to all you talented programers. When
evaluating your program, it would be quite valuable to look at the correlation
between the opening variation evaluation and the actual rate that white wins
that opening against an equal oponent. You can do this in two ways
1) Have 100 engines play against themselves in each opening variation. Then
calculate rate of white success (e.g., 54% or whatever)? So you might get data
that looks like this
Rybka evaluation after 5 minutes actual success rate of variation
.23 53%
-.15 47%
.76 58%
etc. Each engine could then be assigned a score according to how closely its
opening variation evaluations matches actual success rate.
2) The second method is to look at success amongst humans, while statistically
controling for playing level of white and black (this is possible only if you
have a large number of human games across a number of variations)
best
Joseph
____________example of clueless engine evaluation:
Position: 1 e4 d5 2. exd5 Qxd5 3. Nc3 Qa5 4. d4 Nf6 5. Nf3 c6 6. Bd2 Qb6 7) bd3,
bg4
Most engines love position for white:
Shootout results 4 minutes 2 sec (pentium 3) n= 15
White wins only 40%!
(we can at least argue that there is little, if any, advantage for white...of
course more testing is needed)
white black
fritz9 0 1
fruit 2.2.1. 1 0
rybka 1.0 0.5 0.5
toga II 1.1. 0.5 0.5
craft 19.19 0 1
Yace pad 0.5 0.5
aristrarch 4.5 0.5 0.5
spike 1.0a 0 1
ruffian 2.1 0 1
zappa 1.1. 0.5 0.5
tao .5.4 0.5 0.5
List 5.12 1 0
pahraon 2.62 0 1
comet b50 1 0
sos 4 0 1
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.