Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Fair conditions?! * Book Testing Conditions of Michael42

Author: Mike S.

Date: 06:30:02 11/01/03

Go up one level in this thread


On November 01, 2003 at 06:43:29, Rolf Tueschen wrote:

>On November 01, 2003 at 04:13:54, Mike S. wrote:
>(...)
>>Btw. on this occasion, do you have a comment for these results provided by a
>>german computerchess fan:
>>http://www.miko42.de/DasDuell/duellindex.html
>>
>>excerpt:
>>
>>Index Fritz 8.0.23 - Shredder 7.04       F8 - S7.04 % Fritz % Sh. games
>>-----------------------------------------------------------------------
>>01 eigenes Buch - eigenes Buch         131,0 - 145,0 47,5    52,5  276
>>12 eigenes Buch - ohne Buch            143,5 - 151,5 48,6    51,4  295
>>                                                            ======

>(...) 2a) I quote
>(...)
>I de-activate the learn-files and make them write-protected. Already played
>games with the same scores are deleted.

>Now you ask here how people take the astonishing results of Michael42. Answer in
>short:
>
>  - Michael42 himself says that science doesn't interest him
>  - Michael42 plays without activated learn-files
>  - Michael42 deletes certain repeated (formerly already played) games

We have to keep in mind that M42 had a *book test* in mind. He wanted to compare
books in this large test (headline "Das Buch-Duell"). He deactivated both, the
book learning (write protected books) and the position learing of Shredder;
Fritz has none. In general, I don't like the idea of disabling the position
learning, because it removes a major engine feature. But when M42 removed the
doubles (!!) which can happen as a result of this, this compensates for the
disabled position learning, obviously.

He wanted to isolate the "pure" book factor, seems to me.

Ok, what would have happened with learning functions active?

1. Fritz+Fritzbook vs. Shredder+Shredderbook: In both cases, the (same!) GUI
does the book learning. I don't expect that one book benefits significantly more
from learning than the other. Both are large and very good books.

2. Fritz+Fritzbook vs. Shredder without book: Fritz will try to repeat
successfull openings more often, which Shredder (obviously :-)) cannot do by
book learning. But: Shredder has a very effective position learning. When the
Shredder engine has to play the opening itself, it will often change the move
decisions in the same position, due to the position learning stored in previous
games.

Shredder is considered to be one of the engines with the most active and
effective position learning. Some fans wrote they believe Shredder even stores
all evals from completed searches. I've recently played two long matches, all
games from one single *same opening variant* each, which I wanted to explore:
All games were different due to position learning only (the opponent was Hiarcs
which has that feature too)! Of course I cannot predict how the results would
have been with learning on, but I believe Shredder wouldn't do much worse than
it did under M42's conditions. Nevertheless it's astonishing that the difference
is so small in these two matches IMO.

Doubles should always be removed anyway, I just guess with book learning there
are nearly none anymore (maybe except very few exotic book draws which happen
then and when).

>Since the book was designed for the complete package and NOT such a nonsense
>crippled entity. The book is created in close relation to the specifications of
>the engine and the learning file technology. And by nature it is statistically
>proven as making sense - for that specific engine.
>
>Now perhaps the whole spooky nonsense becomes clear. If you cripple the whole
>setting then suddenly you find that the book makes no sense at all. Well done.

Please, the effect of book learning can always be only *a small part* of the
total effect of the book itself. IOW the book is the main thing in the first
place, then book learning is added to improve things even a bit more.

Fritz could use it's book in *both* matches I quoted. Shredder only in the first
one, no book in the second match. This is a much more fundamental difference
than all the learning things.

>(...) Nothing wrong with the private hobby of Michael42, but it's of no use
>for a serious debate about opening books.

He had very good results with a special book he has compiled... I don't see why
this should do worse with learning on. I think it would benefit from the
learning function just like other books do as well. I really think the book
comparison is valid.

Shredder's best result was in the comparison with my Kurzbuch :-) an
experimental book which really wasn't meant as a "normal" opening book. It has a
depth of 6 plies only. But this shows IMO that the Shredder engine must handle
the opening better than Fritz, when both engines have to calculate early.
Fritz 8+Kurzbuch vs. Shredder 7.04+Kurzbuch, 73,5-115,5 (38,9%-61,1%!). With the
own books for both or with Noomen opening variants, Fritz made >46%.

Regards,
Mike Scheidl



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.