Author: Robert Allgeuer
Date: 06:11:52 03/03/04
I have compared several Ruffian version in a kind of qualification tournament
for my YABRL Blitz rating list (see
http://f11.parsimony.net/forum16635/messages/62408.htm for latest published
list).
Conditions:
Time control 300+2, all 3,4 and 5 men EGTB, hash 96MB, ponder off, Athlon
1.1GHz, Win2k, Winboard and WBTM tourney manager 0.60, Elostat 1.1b
Participants:
Ruffian 1.0.1 with 1.0.1 book
Ruffian 2.0.0 with 2.0.0 book
Ruffian Leiden with the Leiden book
Ruffian 2.0.2 with 2.0.0 book
Ruffian 2.1.0 with 2.0.0 book
Ruffian 08.02.2004 (this is a beta version before release before 2.0.2 and
2.1.0) with 2.0.0 book
and as opponents:
Smarthink 0.17a
Gromit 3.8.2
Thinker 4.5b
Crafty 17.14DC
Crafty MPC
Aristarch 4.37
All versions of Ruffian have played matches of 20 games each against each other
and against each opponent (Ruffian 1.0.1 had one duplicate game which was
removed).
Results:
Program Elo + - Games Score Av.Op. Draws
1 Ruffian 08.02.2004 : 2722 39 36 220 61.6 % 2640 37.7 %
2 Ruffian Leiden : 2713 40 35 220 60.2 % 2640 38.6 %
3 Ruffian v2.1.0 : 2703 41 35 220 58.9 % 2641 37.7 %
4 Ruffian v2.0.2 : 2695 42 31 220 57.5 % 2642 45.0 %
5 Ruffian v2.0.0 : 2664 45 31 220 52.7 % 2645 41.8 %
7 Ruffian v1.0.1 : 2650 47 32 219 50.5 % 2646 37.9 %
Observations:
1. This applies of course only to the conditions of this test (Blitz etc.)
2. Ruffian 2.0.0 appears to be stronger than the free Ruffian 1.0.1, although
only a bit. In this test it is 14 ELO points, in my more accurate YABRL rating
list - after more than 800 games each - it is 28 points.
3. Ruffian Leiden and the newer versions (2.0.2, 2.1.0 and 08.02.2004) are
stronger than version 2.0.0. However, they are close to each other and it
appears difficult to determine which of them is indeed the strongest.
4. When looking at the results of version 2.1.0 in more detail, it becomes
apparent that it scores consistently less than the other Ruffian versions
(except 1.0.1), but "saves" its high rating only by scoring high in the direct
matches against Ruffian 2.0.2 and 08.02.2004. Nevertheless 2.1.0 appears to be
the weakest of the new Ruffian version in matches against other non-Ruffian
engines.
5. From the characteristics of its results it becomes apparent that 08.02.2004
is a (late) beta version of Ruffian 2.0.2 (and not 2.1.0). It would be highly
interesting, whether this version is in fact identical to 2.0.2 (the measured 27
points difference in strength are within the error margin) or there were some
changes made before release of 2.0.2, which might have decreased 2.0.2's playing
strength.
6. The Leiden version seems to be one of the strongest. Version 2.0.2 is a
bug-fix version of 2.0.0 and some 30 points stronger than 2.0.0. If I had a
wish, I would ask for a bug-fix Leiden version; that one would most probably be
the strongest Ruffian of all.
Robert
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.