Author: Vasik Rajlich
Date: 04:09:19 01/12/06
Go up one level in this thread
On January 12, 2006 at 07:01:06, Albert Silver wrote: >On January 12, 2006 at 06:58:21, Vasik Rajlich wrote: > >>On January 12, 2006 at 06:14:11, Uri Blass wrote: >> >>>On January 12, 2006 at 06:07:47, Vasik Rajlich wrote: >>> >>>>On January 11, 2006 at 23:22:09, Albert Silver wrote: >>>> >>>>>Since I am not fond of random openings to test something as specific as which >>>>>settings are best, I ran two full matches using the Noomen Select positions >>>>>against Fruit 2.2 to test the Neutral and Optimistic settings of Rybka Beta 9. >>>>> >>>>>For those who aren't familiar with them, the Noomen Select positions are a set >>>>>similar to those of the Nunn2 set, except there are 40 (leading to 80-game >>>>>matches) and they were chosen by well-known opening book designer Joeroen >>>>>Noomen. >>>>> >>>>>The results were interesting, and showed a marked improved performance for the >>>>>Optimistic settings. >>>>> >>>>>Match conditions were: >>>>> >>>>>Athlon64 3500+ (Venice) + 1GB Ram >>>>>Time control: 10mins + 2sec increment >>>>>Ponder off; 256MB hashtables each >>>>>Noomen Select positions >>>>> >>>>>Noomen-RybkaB9Neutral-Fruit22 2006 >>>>> >>>>>1 Rybka 1.01 Beta 9 32-bit 2900 +6 +34/-21/=25 58.13% 46.5/80 >>>>>2 Fruit 2.2 2850 -6 +21/-34/=25 41.88% 33.5/80 >>>>> >>>>>The GUI said Rybka performed 56 Elo over Fruit 2.2 using the Neutral settings. >>>>> >>>>>Noomen-RybkaB9Optimistic-Fruit22 2006 >>>>> >>>>>1 Rybka 1.01 Beta 9 32-bit 2900 +52 +36/-13/=31 64.38% 51.5/80 >>>>>2 Fruit 2.2 2850 -52 +13/-36/=31 35.63% 28.5/80 >>>>> >>>>>Here the GUI said Rybka performed 102 Elo over Fruit 2.2 using the Optimistic >>>>>settings. >>>>> >>>>>In order to gather more data I'll run a shorter Nunn2 match (only 40 games) >>>>>against Gambit Fruit 4bx, which had been the first Beta's toughest opponent in >>>>>the Nunn2 set aside from Fruit 2.2 in my testing. >>>>> >>>>>The rating of 2850 for Fruit was taken from the latest SSDF list, and Rybka's >>>>>2900 was chosen arbitrarily. >>>>> >>>>> Albert >>>> >>>>Albert, >>>> >>>>thanks for the data. As I told you in the email, it may be that the neutral >>>>setting is good in self-play while the optimistic setting is better for getting >>>>bigger scores. >>>> >>>>Vas >>>> >>>>ps. Iweta is now watching over my shoulder and gives her greetings - and says >>>>that after Budapest, Rio is the best :) >>> >>>It is also possible that optimistic is simply better against weaker players when >>>neutral is better against stronger players or players of the same strength. >>> >>>The way to test it is to give fruit more time(in case of ponder off) or better >>>hardware in case of ponder on) and to test both optimistic and neutral. >>> >>>Uri >> >>Another possibility (which we will test with Beta 10) is that optimistic is >>better in better positions. I will try an "adaptive" approach, where if the >>previous iteration gave a score above a section threshold, an optimistic setting >>is used. >> >>Vas > > >Maybe, maybe not. Note that in the above results, Optimistic only won two more >games than Neutral. The biggest difference is that it lost far fewer. > > Albert Indeed, maybe I will look at that when all the data is ready. The adaptive approach is something which makes some sense to me. It will work if it's true that in better positions, you are more likely to find a move which becomes stronger with a deeper search. Yes, it's a mouthful - I hope it makes sense :) Vas
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.