Author: Joachim Rang
Date: 08:56:10 07/26/02
Go up one level in this thread
On July 26, 2002 at 06:15:06, Shaun Brewer wrote: >On July 25, 2002 at 14:40:06, Joachim Rang wrote: > >>I'm going to make some automatches between amateur engines. As this is the first >>time I try to make this, I'd be glad to get some advices. >> >>I will use a PIII-Notebook at 1.1 GHz with 256 MB RAM. I plan to make not a >>tournament but one to one matches with much games (a few hundreds) to get >>significant results. >> >>What are fair and reasonable testconditions? > >Try to reduce the number of background activities. On one machine I run Norton >System Works which seems to ruin results. So I don't use this for matches. > >> >>I will run the match on PIII 1.1 GHz and 256 MB RAM under WinXP. I think its >>reasonable to give every engine 80 MB Hashtablesize. I still don't know which >>GUI I shall use. I tested Arena and it worked, but with some problems. With >>Winboard I don't know how to make automatches, maybe someone can help? >> >>First engine will be Yace 0.99.56, which I consider the strongest amateurengine >>today. >>Which engine can compete with Yace? >> > >Crafty - I have so far found 17.10 to be stronger than 18.15 > >Tests on Tbird Athlon 850 (120+2) > >Individual statistics: > >(1) wcrafty-17.10 : 500 (+166,=246,- 88), 57.8 % >(2) Crafty-18.15 : 500 (+ 88,=246,-166), 42.2 % > > Program Score % Av.Op. Elo + - >Draws > > 1 wcrafty-17.10 : 289.0/500 57.8 2373 2427 27 19 >49.2 % > 2 Crafty-18.15 : 211.0/500 42.2 2427 2373 19 27 >49.2 % > >Tests on Tbird Athlon 850 (300+5) > >Individual statistics: > >(1) wcrafty-17.10 : 500 (+159,=231,-110), 54.9 % >(2) Crafty-18.15 : 500 (+110,=231,-159), 45.1 % > > Program Score % Av.Op. Elo + - >Draws > > 1 wcrafty-17.10 : 274.5/500 54.9 2383 2417 29 20 >46.2 % > 2 Crafty-18.15 : 225.5/500 45.1 2417 2383 20 29 >46.2 % > >Tests on Tbird Athlon 850 (900+15) > >Still running something like +100 -40 =100 > >Tests on PIII 1332mhz (1800+30) > >Still running (when machine not in use) too few games to be conclusive > >Individual statistics: > >(1) wcrafty-17.10 : 52 (+ 23,= 20,- 9), 63.5 % >(2) Crafty-18.15 : 52 (+ 9,= 20,- 23), 36.5 % > > >>I think I will run the games with 10 minutes and 10 s per move for each engine. >>Is this to short? > >No correct answer to this one. > >Some engines may be poor at blitz relative to their standard performance, the >reverse or consistent. > >The longer the time control the better the quality of the games but a match to >provide statistially relevent results could take months. > >> >>Another question is, which opening books I shall use. Yace has an own book as >>most of the other engines. But I don't know how good they are, SOS for example >>comes only with a very little own book, which will be a handicap. Does someone >>know a "neutral" but large book, which will run with the most engines? > >For your experiment I would use the book provided/recomended by the author >initially. If you then test with a different book and this produces better >results share your findings with the author. > >Have fun > >Shaun > >> >>So far today, I'd appreciate any suggestions from you. Interesting results for crafty. Maybe I choose to make the first match between crafty 17.10 and Yace 0.99.56. Which opening book you used for your matches (don't say from Hyatts FTP) and which configuration file?
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.