Author: Dann Corbit
Date: 13:29:00 02/21/05
Go up one level in this thread
On February 21, 2005 at 15:15:49, Tord Romstad wrote: >Hi Dann! > >On February 21, 2005 at 13:40:37, Dann Corbit wrote: > >>This is on the 2.2 GHz 64 bit computer (which had fewer problems than the slower >>machine). The first set is the raw data, and the second set has had all >>duplicate games removed. >> >> Program Elo + - Games Score Av.Op. Draws >>1 Polyglot fruit-book : 2553 31 35 372 59.9 % 2483 18.3 % >>2 Polyglot glaurung-book : 2540 32 35 372 57.5 % 2487 16.7 % >>3 Polyglot glaurung-no-book: 2481 32 34 372 46.2 % 2507 16.7 % >>4 Polyglot fruit-no-book : 2427 40 29 372 36.3 % 2525 14.0 % >> >> Program Elo + - Games Score Av.Op. Draws >>1 Polyglot fruit-book : 2550 32 37 348 59.3 % 2484 17.5 % >>2 Polyglot glaurung-book : 2536 33 35 363 56.7 % 2489 16.5 % >>3 Polyglot glaurung-no-book: 2483 33 36 343 46.9 % 2505 16.9 % >>4 Polyglot fruit-no-book : 2430 41 30 354 36.9 % 2524 12.7 % >> >>It seems that Fruit may benefit from a book slightly more than glaurung. > >This is a big surprise to me. Glaurung has a very primitive development >eval, and no mobility at all except some very simple stuff for bishops. I >thought it would be completely unable to find good opening moves on >its own. > >Perhaps the explanation simply is that stronger engines tend to profit >more from playing with a book, as Sandro has suggested. That is highly counter-intuitive to me. I can understand that strong engines would not be as likely to "get lost" when playing gambits and things of that nature, but they do not have as much headroom to benefit from. Consider an engine that plays 1000 Elo. Give it a book 100 ply deep with no errors and all moves of top engine SSDF or SuperGM quality. Give the same book to Shredder. I cannot imagine how Shredder will benefit more. I think (paradoxically) that the opposite is more logical to me. For instance, the books that the professional programs have will be realistically many orders of magnitude better than the auto-generated sort that I can produce. And so if they get a 100 Elo boost, I imagine that a weaker program will get far more from it because they would have played stupid moves early on. >It could also >be related to the relative strengths and weaknesses of the two programs. >When I run test matches between Fruit and Glaurung, I have noticed that >the Glaurung's scores most of its wins in the middle game, while Fruit >tends to win most endgames. Perhaps playing without a book reduces >Fruit's chances of reaching the endgame. I also find it surprising, since Fruit has a very sophisicated mobility calculation. >Thanks for doing this experiment, and for your nice choice of engines >for the experiment! :-) I prefer open source engines for my tests (if possible). This test required UCI because I was also interested in testing the Polyglot book feature at the same time.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.