Author: Robert Allgeuer
Date: 10:47:49 08/27/04
This is an extension of the Fruit 1.5 parameter test that I have run recently
(see past post on <http://f11.parsimony.net/forum16635/messages/69839.htm>. It
was also posted in CCC, but for some reasons it does not show up in the search
engine)
Method:
=======
The test consisted of a round robin tournament of several configurations of
Fruit 1.5 and a set of reference engines (please note that this test was run
before the El Chinito case has become public).
The Nunn 1 starting positions were used; for each pairing each engine had to
play both sides, resulting in 20 games for each pairing and slightly more than
5000 games overall.
The tournament results have been analysed with Elostat and a corresponding
rating table has been calculated.
Platform, Tools and Settings:
=============================
Athlon XP 2400+
1.1 GB RAM
Windows XP
Elostat 1.1b
Arena 1.08
Time Control: 5min + 2sec
Ponder off
EGTBs enabled when supported
64MB Hash
Participants:
=============
Ten different configurations of Fruit 1.5, including the default settings, eight
settings with exactly one UCI-parameter modified each and one setting with a
combination of 3 modified parameters:
Fruit v1.5def: Fruit 1.5 with the default parameter setting
Fruit v1.5nmalways: nullmove search is tried always (instead of in the fail-high
case only)
Fruit v1.5noetc: ETC disabled
Fruit v1.5ppushext: pawn push extension (7th rank) enabled
Fruit v1.5nosinglerep: single reply extension disabled
Fruit v1.5noqchecks: quiescence search does not include checking moves
Fruit v1.5nmR2: nullmove reduction set to 2 instead of the default 3
Fruit v1.5qchknm: quiescence search considers checking moves only after a
nullmove
Fruit v1.5matvjr: alternative settings for piece values by J. Rang
Fruit v1.5comb1: combination of nullmove always, ETC disabled and pawn push
extensions enabled; the three parameter settings that each have resulted in a
higher result than the default settings (albeit in some cases only by a very
tiny margin)
plus 13 other engines as reference opponents.
Results:
========
Program Elo + - Games Score Av.Op. Draws
1 Ruffian v1.01 : 2698 24 38 440 70.7 % 2545 21.8 %
2 List v5.12 : 2664 26 35 440 66.2 % 2547 23.0 %
3 El Chinito v3.25 : 2648 26 33 440 64.1 % 2548 24.1 %
4 Gothmog v0.4.8 : 2604 29 31 440 57.7 % 2550 18.6 %
5 Fruit v1.5nmalways : 2598 30 29 440 56.8 % 2550 22.3 %
6 Fruit v1.5noetc : 2577 31 28 440 53.8 % 2551 21.1 %
7 Fruit v1.5comb1 : 2576 31 28 440 53.6 % 2551 21.8 %
8 Fruit v1.5ppushext : 2573 32 27 440 53.1 % 2551 24.3 %
9 Fruit v1.5def : 2568 32 28 440 52.4 % 2551 21.6 %
10 Fruit v1.5qchknm : 2560 33 27 440 51.1 % 2552 21.8 %
11 Fruit v1.5matvjr : 2557 33 26 440 50.8 % 2552 23.4 %
12 Fruit v1.5noqchecks : 2557 33 28 440 50.7 % 2552 19.5 %
13 Fruit v1.5nosinglerep : 2554 33 25 440 50.3 % 2552 27.5 %
14 AnMon v5.21 : 2550 26 33 440 49.7 % 2552 24.3 %
15 SoS4 : 2543 26 32 440 48.6 % 2553 24.5 %
16 Ktulu v5.0 : 2541 25 32 440 48.3 % 2553 29.8 %
17 Fruit v1.5nmR2 : 2534 27 32 440 47.3 % 2553 25.0 %
18 Amyan v1.592 : 2533 28 32 440 47.2 % 2553 21.6 %
19 Yace Paderborn : 2513 30 30 440 44.1 % 2554 19.1 %
20 Ufim v5.00 : 2465 33 27 440 37.2 % 2556 22.5 %
21 Frenzee v1.59 : 2441 36 26 440 33.9 % 2557 19.5 %
22 Patzer v3.61 : 2427 38 25 440 32.0 % 2558 19.5 %
23 Sjeng v12.13 : 2415 39 25 440 30.5 % 2558 19.5 %
Essentially all these parameters have pretty much no impact on Fruit´s playing
strength with the probable exceptions of:
- Enabling "nullmove always" probably increases playing strength
- Setting nullmove reduction to 2 probably decreases playing strength
Interestingly the combined setting (comb1) scored lower than the pure nmalways
setting, but maybe this can be blamed on statistics.
Conclusion:
===========
Generally the impact of the different UCI parameter settings on Fruit´s playing
strength is comparatively small, in the end all results still fall within the
error margins of ~30.
I personnally am a bit surprised that enabling/disabling the extensions makes
pretty much no difference, and would be interested in views as to why this would
be the case.
I would have also expected a bigger impact of the modified piece value settings
and of disabling/enabling checks in the quiescence search.
Robert
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.