Author: William Penn
Date: 16:23:26 09/28/03
-------------------------------
Shredder 7 - erratic analysis
-------------------------------
Introduction:
Three configurations (variables: GUI & hash size) were tested as described
below. The tests were fairly simple: the starting position was setup, Infinite
Analysis mode was turned on, then the analysis results were copied after 3+
hours runtime. Each test was repeated to give an idea about variance. If
everything else were equal, we except the different runs with a particular test
configuration to be identical or very similar - however the differences are
LARGE! I've done my best, but cannot figure out what is causing them. Only one
position is tested below, but I have noticed that other positions also show
large analysis variances. These tests are with the Shredder 7.04 engine. I have
also run some tests with the Shredder 7 engine and it behaves the same - giving
erratic analysis. I'm familiar with the usual variations from op system paging,
but believe something else may be involved here. It is probably caused by the
Shredder 7/7.04 engine because it happens with both the Shredder 7 and Classic
Shredder 6 GUI's. So, is it an engine bug, or what?
---------------------------------------------------------------------------
My computer system, and the test variables:
Computer: 2.0G Athlon XP 2400+, 1G RAM (992MB free), 120G hd (80% free)
Op system: Windows XP Home (Service Pack 1), DirectX 9.0
GUI's used in these tests:
Shredder 7 GUI (probably same as Fritz 8 GUI), March 5 2003 update
and the Classic Shredder 6 GUI, installed from the Shredder 7 CD
All tests were run in Infinite Analysis mode.
All tests use the Shredder 7.04 engine:
the .eng version was used with the Shredder 7 GUI;
the UCI version was used with the Classic Shredder 6 GUI.
The position used in all these tests, fen:
6k1/3q1pbp/p1Np2p1/1p1P2PP/5Q2/4rP2/PPP1r3/1KR4R w - - 0 1
That is an "unclear & active" middlegame position.
Since this is a middlegame position, opening books are not involved.
The tablebases were activated, but this position doesn't access them.
---------------------------------------------------------------------------
Configuration 1: 512MB hash, Shredder 7 GUI
Run1:
analysis line eval depth time nodes kN/s
26.hxg6 fxg6 27.Qh4 Be5 28.Rcf1 1.36 17/46 00:12:33 263891kN 350kN/s
26.hxg6 fxg6 27.Qh4 Bh8 28.Qb4 1.25 18/46 00:34:29 723737kN 349kN/s
26.hxg6 fxg6 27.Qh4 Be5 28.Rhf1 1.07 19/49 01:31:22 1904411kN 347kN/s
26.h6 Bh8 27.Qb4 Rxf3 28.Qa5 1.31 19/51 02:11:15 2730349kN 346kN/s
26.h6 Bh8 27.Qb4 Rxf3 28.Qa5 1.31 20/51 02:38:05 3288857kN 347kN/s
(stopped, after the 21/21 level begins calculations at 03:35:45)
Run2 (reset computer):
26.hxg6 fxg6 27.Qh4 Be5 28.Rcf1 1.34 17/39 00:07:33 154429kN 339kN/s
26.hxg6 fxg6 27.Qh4 Be5 28.Rcf1 1.36 18/47 00:17:20 353755kN 338kN/s
26.hxg6 fxg6 27.Qh4 Be5 28.Rcf1 1.34 19/46 00:32:47 667255kN 338kN/s
26.hxg6 fxg6 27.Qh4 Bh8 28.Qb4 1.22 20/50 01:32:16 1872683kN 338kN/s
(still calculating after 5+ hours with no further output, so stopped)
---------------------------------------------------------------------------
Configuration 2: 784MB hash (maximum accepted by GUI), Shredder 7 GUI
Run1:
26.hxg6 fxg6 27.Qh4 Be5 28.Rcf1 1.13 17/45 00:17:39 367492kN 345kN/s
26.hxg6 fxg6 27.Qh4 Be5 28.Rcf1 1.26 18/51 00:42:28 875614kN 344kN/s
26.hxg6 fxg6 27.Qh4 Be5 28.Rcf1 1.16 19/49 01:13:54 1517996kN 341kN/s
26.hxg6 fxg6 27.Qh4 Be5 28.Rcf1 1.12 20/51 02:39:29 3261028kN 341kN/s
(stopped after 3.5 hours, seemed too sluggish)
---------------------------------------------------------------------------
Configuration 3: 512MB hash, Classic Shredder 6 GUI
Run1:
depth time eval analysis line nodes kN/s
17/47 16:19 +1.32 26.hxg6 fxg6 27.Qh4 Be5 (338.238.247) 345.4
18/48 29:37 +1.30 26.hxg6 fxg6 27.Qh4 Be5 (608.766.698) 342.4
(botched it, my fault, stopped)
Run2 (reset computer):
17/45 15:56 +1.33 26.hxg6 fxg6 27.Qh4 Bh8 (331.174.749) 346.1
18/47 26:07 +1.33 26.hxg6 fxg6 27.Qh4 Bh8 (541.920.888) 345.7
19/47 64:42 +1.17 26.hxg6 fxg6 27.Qh4 Bh8 (1.331.269.680) 342.8
20/55 198:42 +1.05 26.hxg6 fxg6 27.Qh4 Be5 (4.076.208.773) 341.8
20/55 239:20 +1.06++ 26.h6 Bh8 27.Qb4 f5 (4.900.234.342) 341.2
20/55 277:10 +1.26 26.h6 Bh8 27.Qb4 Kf8 (5.669.420.818) 340.8
Run3 (reset computer):
17/46 15:15 +1.33 26.hxg6 fxg6 27.Qh4 Bh8 (316.684.504) 345.8
18/46 32:48 +1.23 26.hxg6 fxg6 27.Qh4 Bh8 (675.717.344) 343.1
19/50 86:21 +1.17 26.hxg6 fxg6 27.Qh4 Bh8 (1.773.284.750) 342.2
19/50 115:48 +1.18++ 26.h6 Bh8 27.Qb4 f6 (2.376.172.077) 341.9
19/50 132:10 +1.29 26.h6 Bh8 27.Qb4 Kf8 (2.710.143.664) 341.7
20/49 163:09 +1.25 26.h6 Bh8 27.Qb4 Rxf3 (3.338.854.412) 341.0
21/55 274:16 +1.44 26.h6 Bh8 27.Qb4 f6 (5.542.118.533) 336.7
---------------------------------------------------------------------------
Notes & Comments:
Lower levels of analysis data (less than 17/xx) were discarded, and analysis
lines were also shortened to simplify this presentation.
Is 26.h6 the best move in this position? Probably yes, in my personal opinion,
although it's somewhat unclear. What matters is that Shredder 7.04 apparently
thinks so if you let it calculate long enough - but not always, it's
inconsistent. Sometimes it doesn't discover that move, and I have been unable to
identify the variable(s) causing this problem.
Configuration 1: Run1 found 26.h6 in a little over 2 hours at level 19/51,
however Run2 hadn't found it after over 5 hours runtime!?
Configuration 2: Run1 didn't find 26.h6 after 3.5 hours, and was acting sluggish
so was aborted and not attempted again.
Configuration 3: Run1 was botched (my fault) and aborted. Run2 found 26.h6 after
almost 4 hours at level 20/55, then Run3 found it in less than 2 hours at level
19/50!?
The biggest problem when attempting tests such as these is that the user has no
control over the analysis tree, no open window into the engine's ongoing
thinking. So we must accept whatever the software decides to spit out, according
to its own internal design & criteria. Unfortunately that appears to be an
unreproducible timetable. We just cannot predict it very well. And without
predictability it is difficult (practically impossible) to design good
comparative tests for this system/engine, such as to determine the optimum hash
size.
Re the Shredder 7 GUI:
It's fairly easy to crash the Shredder 7 GUI. Just enter a new game, edit the
game data, then try to save it in the current database. It freezes and may
require a computer restart to recover completely. [So I've been using Chessbase
8 for that purpose, which works OK.] If I change the hash table size and
tablebase cache size simultaneously, and it may hang the computer. The shortcut
that switches between the chessboard and database windows often fails too (goes
nowhere, become inactive). There are also many "fatal exceptions" where Windows
closes the GUI automatically, and those error messages suggest that the C+ code
(or whatever, I can't remember exactly) was not compiled correctly. So to use
this GUI requires considerable tender loving care. The user must limit
operations, trying to sidestep buggy sequences that crash the system. I can only
speculate that it may need to be recoded & recompiled for the latest SP1 version
of Windows XP.
The other GUI:
In the meantime, the Classic Shredder 6 GUI is reasonably friendly. I like it,
and it appears to work OK in general. The main disadvantage for me is that it
understands PGN format, but not Chessbase format (.cbh), so I must convert
databases or individual games to PGN so it can access them. The Shredder 7 GUI
can also read PGN format, so they are able to access the same database file (but
not simultaneously). That's important to me, because it would be difficult to
give up the considerable advantages of the Shredder 7 GUI entirely.
--
WP
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.