Author: STEPHEN A. BOAK
Date: 14:05:44 01/10/99
Howard Exner has an interesting article entitled 'Rebel 10 in the Endgame',
posted on here on CCRC. His writeup indicates he gave 7 full minutes for
analysis of each position.
My subject is only his first position:
EXNER EPD Suite, Position #01
5k2/8/7P/8/8/8/K6P/1B6 w - - id END01; bm Bh7;
Howard shows the following test results for various Rebel software versions:
Endgame Test Suite - Rebel 8,9 and 10
Machine - AMD K-233 with 60 MB Hash running in Dos
8-D is Rebel 8 default | 9-D is Rebel 9 default | 10-NC is Rebel 10 with
Combination Off 10-ST4A is Rebel 10 with Combination off, AG=Strong, Selection =
4 and Play Style = Active
X denotes not solved in under 7 minutes | am means "avoid move"
POS Move
1 8-D 9-D 10-NC 10-ST4A 1 Bh7
0:52 0:48 1:03 1:20
I analyzed the same position for approximately 12 hours, using ECTool and
special Rebel 10 analysis engine for ECTool. This software combination only
allows up to 28MB maximum (which I used). I used a Pentium 200 (not MMX)
machine.
Here are my analysis results:
Rebel Engine for ECTool. (c) Ed Schröder
Engine version : REBEL 10
Hash table size : 28 Mb
Analysis mode : Analyzing next move
Refresh interval : 1000 ms
Game begin
00:01 08.01 5.99 Bb1-h7 Kf8-f7 Ka2-b2 Kf7-f8 Kb2-c3 Kf8-e7 Bh7-e4
00:02 10.00 5.29 Bb1-h7 Kf8-e7 Ka2-b3 Ke7-f7 Kb3-c4 Kf7-f8 Bh7-e4
00:02 10.01 5.36 Bb1-e4 Kf8-g8 Ka2-b3 Kg8-h8 Kb3-c4 Kh8-g8 Kc4-d4
00:03 11.01 6.23 Bb1-h7 Kf8-e7 Ka2-b3 Ke7-f6 h2-h4 Kf6-f7 h4-h5
00:06 12.00 5.36 Bb1-h7 Kf8-e7 Ka2-b2 Ke7-f6 Bh7-e4 Kf6-f7 Kb2-c3
00:06 12.01 5.42 Bb1-e4 Kf8-g8 Ka2-b3 Kg8-h8 Kb3-c4 Kh8-g8 h2-h4
00:09 13.00 5.42 Bb1-e4 Kf8-g8 Ka2-b3 Kg8-h8 Kb3-c4 Kh8-g8 h2-h4
00:13 14.00 5.43 Bb1-e4 Kf8-g8 Ka2-b3 Kg8-h8 Kb3-c4 Kh8-g8 Kc4-d4
00:13 14.01 5.43 Bb1-h7
00:16 14.01 6.37 Bb1-h7 Kf8-e7 h2-h4 Ke7-f6 Ka2-b3 Kf6-f7 Kb3-c4
00:24 15.00 5.45 Bb1-h7 Kf8-e7 h2-h4 Ke7-f6 Ka2-b3 Kf6-f7 Kb3-c3
00:26 15.01 5.50 Bb1-e4 Kf8-g8 Ka2-b3 Kg8-h8 h2-h4 Kh8-g8 Kb3-c4
00:37 16.01 5.46 Bb1-h7
00:39 16.01 6.41 Bb1-h7 Kf8-e7 Ka2-b3 Ke7-f7 h2-h4 Kf7-f6 Kb3-c4
00:54 17.00 6.38 Bb1-h7 Kf8-f7 Ka2-b3 Kf7-e7 Kb3-c4 Ke7-f7 h2-h4
01:13 18.00 5.50 Bb1-h7 Kf8-f7 Ka2-b3 Kf7-e7 Bh7-e4 Ke7-f7 Kb3-c4
01:19 18.02 5.51 h2-h4 Kf8-g8 Ka2-b2 Kg8-h8 Kb2-c3 Kh8-g8 Kc3-d4
01:33 19.00 5.68 h2-h4 Kf8-g8 Ka2-b2 Kg8-h8 Kb2-c3 Kh8-g8 Kc3-d4
01:39 19.01 5.68 Bb1-h7
01:50 19.01 6.48 Bb1-h7 Kf8-f7 Ka2-b3 Kf7-e7 Kb3-c4 Ke7-f7 h2-h4
02:35 20.00 6.72 Bb1-h7 Kf8-f7 Ka2-b3 Kf7-e7 Kb3-c4 Ke7-f7 h2-h4
03:20 21.00 6.72 Bb1-h7 Kf8-f7 h2-h4 Kf7-f8 Ka2-b3 Kf8-f7 Kb3-c4
46:47 22.00 13.42 Bb1-h7 Kf8-f7 h2-h4 Kf7-f6 Ka2-b3 Kf6-f7 Kb3-c4
49:57 23.00 13.56 Bb1-h7 Kf8-f7 h2-h4 Kf7-f6 Ka2-b3 Kf6-f7 Kb3-c4
Note that the times shown by ECTool only are minutes and seconds. After a full
hour, the minutes may drop from 59 min to 1 min. Therefore I can not tell
exactly how many hours (plus the shown minutes and seconds) elapsed for the last
two or three analysis lines. I quit analyzing after approx 12 hours, and the
last two lines had already finished, as shown above.
My question for Howard Exner is this--what constitutes a solved position in his
endgame test suite test of Rebel versions?
The last shown best move at the end of 7 minutes of analysis?
The first time posted, for the last shown best move at the end of 7 minutes,
after which the added lines did not change the best move?
Do you use the PV score to determine if a solution is 'understood' by the
program?
In my analysis results, for example, ECTool/Rebel 10 waffled between a total of
3 different moves, over 21 ply levels, showing the score in the 5 to 7 point
range (White has B+2P; Black a King only). Even on ply 19.00, the PV showed h4
as the best move (incorrect solution). On ply 22, the program finally
calculated a PV score of 13+ for White, indicating it finally saw a winning pawn
promotion.
Howard tested the ability of Rebel software versions to find endgame solutions
for his EPD suite of 30 positions, during typical real life game analysis times
(he used 7 full min per position). For this purpose his test provides useful
feedback.
A test of limited time duration will not always allow a program to finalize a
best move (waffling may still occur, as analysis continues).
However, in a constructed test position where there is a specific solution,
until the program sees the 'gain' from the best line of play (either mate or
substantial material increase), it may continue to waffle, depending on how much
time it has in an actual game to make a decision. It makes me wonder if the
program really 'has' or 'would' solve the problem in actual play. Perhaps, for
example, a program might waffle back and forth, each turn to play, taking up to
7 minutes to find best moves but never fully understanding them enough to drive
the position to a complete solution.
This is food for thought--any others with ideas on the subject?
--Steve
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.