Author: José de Jesús García Ruvalcaba
Date: 11:23:56 08/27/04
Go up one level in this thread
On August 27, 2004 at 13:47:49, Robert Allgeuer wrote: > >This is an extension of the Fruit 1.5 parameter test that I have run recently >(see past post on <http://f11.parsimony.net/forum16635/messages/69839.htm>. It >was also posted in CCC, but for some reasons it does not show up in the search >engine) > > > >Method: >======= > >The test consisted of a round robin tournament of several configurations of >Fruit 1.5 and a set of reference engines (please note that this test was run >before the El Chinito case has become public). > >The Nunn 1 starting positions were used; for each pairing each engine had to >play both sides, resulting in 20 games for each pairing and slightly more than >5000 games overall. > >The tournament results have been analysed with Elostat and a corresponding >rating table has been calculated. > > > >Platform, Tools and Settings: >============================= > >Athlon XP 2400+ >1.1 GB RAM >Windows XP > >Elostat 1.1b >Arena 1.08 > >Time Control: 5min + 2sec >Ponder off >EGTBs enabled when supported >64MB Hash > > > >Participants: >============= > >Ten different configurations of Fruit 1.5, including the default settings, eight >settings with exactly one UCI-parameter modified each and one setting with a >combination of 3 modified parameters: > >Fruit v1.5def: Fruit 1.5 with the default parameter setting >Fruit v1.5nmalways: nullmove search is tried always (instead of in the fail-high >case only) >Fruit v1.5noetc: ETC disabled >Fruit v1.5ppushext: pawn push extension (7th rank) enabled >Fruit v1.5nosinglerep: single reply extension disabled >Fruit v1.5noqchecks: quiescence search does not include checking moves >Fruit v1.5nmR2: nullmove reduction set to 2 instead of the default 3 >Fruit v1.5qchknm: quiescence search considers checking moves only after a >nullmove >Fruit v1.5matvjr: alternative settings for piece values by J. Rang >Fruit v1.5comb1: combination of nullmove always, ETC disabled and pawn push >extensions enabled; the three parameter settings that each have resulted in a >higher result than the default settings (albeit in some cases only by a very >tiny margin) > >plus 13 other engines as reference opponents. > > > >Results: >======== > > Program Elo + - Games Score Av.Op. Draws > > 1 Ruffian v1.01 : 2698 24 38 440 70.7 % 2545 21.8 % > 2 List v5.12 : 2664 26 35 440 66.2 % 2547 23.0 % > 3 El Chinito v3.25 : 2648 26 33 440 64.1 % 2548 24.1 % > 4 Gothmog v0.4.8 : 2604 29 31 440 57.7 % 2550 18.6 % > 5 Fruit v1.5nmalways : 2598 30 29 440 56.8 % 2550 22.3 % > 6 Fruit v1.5noetc : 2577 31 28 440 53.8 % 2551 21.1 % > 7 Fruit v1.5comb1 : 2576 31 28 440 53.6 % 2551 21.8 % > 8 Fruit v1.5ppushext : 2573 32 27 440 53.1 % 2551 24.3 % > 9 Fruit v1.5def : 2568 32 28 440 52.4 % 2551 21.6 % > 10 Fruit v1.5qchknm : 2560 33 27 440 51.1 % 2552 21.8 % > 11 Fruit v1.5matvjr : 2557 33 26 440 50.8 % 2552 23.4 % > 12 Fruit v1.5noqchecks : 2557 33 28 440 50.7 % 2552 19.5 % > 13 Fruit v1.5nosinglerep : 2554 33 25 440 50.3 % 2552 27.5 % > 14 AnMon v5.21 : 2550 26 33 440 49.7 % 2552 24.3 % > 15 SoS4 : 2543 26 32 440 48.6 % 2553 24.5 % > 16 Ktulu v5.0 : 2541 25 32 440 48.3 % 2553 29.8 % > 17 Fruit v1.5nmR2 : 2534 27 32 440 47.3 % 2553 25.0 % > 18 Amyan v1.592 : 2533 28 32 440 47.2 % 2553 21.6 % > 19 Yace Paderborn : 2513 30 30 440 44.1 % 2554 19.1 % > 20 Ufim v5.00 : 2465 33 27 440 37.2 % 2556 22.5 % > 21 Frenzee v1.59 : 2441 36 26 440 33.9 % 2557 19.5 % > 22 Patzer v3.61 : 2427 38 25 440 32.0 % 2558 19.5 % > 23 Sjeng v12.13 : 2415 39 25 440 30.5 % 2558 19.5 % > > > >Essentially all these parameters have pretty much no impact on Fruit´s playing >strength with the probable exceptions of: >- Enabling "nullmove always" probably increases playing strength >- Setting nullmove reduction to 2 probably decreases playing strength > >Interestingly the combined setting (comb1) scored lower than the pure nmalways >setting, but maybe this can be blamed on statistics. > > > >Conclusion: >=========== > >Generally the impact of the different UCI parameter settings on Fruit´s playing >strength is comparatively small, in the end all results still fall within the >error margins of ~30. > >I personnally am a bit surprised that enabling/disabling the extensions makes >pretty much no difference, and would be interested in views as to why this would >be the case. > Just speculation on my part, but probably because you played with a fast time control. José. >I would have also expected a bigger impact of the modified piece value settings >and of disabling/enabling checks in the quiescence search. > >Robert
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.