Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Most brilliant novelty from cct7 Witchess-Arasan

Author: Peter Berger

Date: 06:29:58 02/15/05

Go up one level in this thread


On February 15, 2005 at 06:46:04, Arturo Ochoa wrote:

>On February 15, 2005 at 02:56:54, Peter Berger wrote:
>
>>On February 14, 2005 at 20:08:42, Arturo Ochoa wrote:
>>
>>>On February 14, 2005 at 19:54:03, Peter Berger wrote:
>>>
>>>>Yes, you opposed this point of view multiple times before in discussions
>>>>with Uri , but I think you never managed to score.
>>
>>>Of, I have managed to score several games during the year 2004.
>>
>>That's just a misunderstanding, because I worded badly :). My "scoring" only
>>applied to the discussions, not to the quality of your work on the opening book.
>>
>>Btw - it's not trivial to think of a good and practical experiment to setup to
>>show who is right.
>>
>>Peter
>
>Well, I have already done such an experiment and I know who will give the
>easiest point. It is great because it will mean easy points.
>
>However, I would not spend again valuable time repeating the same nonsense
>experiment.
>
>Arturo.

I think you can't really do such a test properly on your own. And it would be so
extremely time-consuming to do in a realistic way, that I doubt anyone has ever
really done it.

You mentioned the major problem I see in another post, learning as done by a
book author. This is a factor that has to be taken into account.

If you take your opening book as prepared for some major event, it is probably
nearly 100% deterministic at start of some round. If you run it against an
automatically learning book in longer matches to get a measure for the quality
of your work, it will get beaten badly. It might do well in the first few games,
until the opponent finds some hole ( which in this case means just some line
where it can beat "your" engine) - then it will repeat it in the following
games.

This is not a realistic test of what would happen in a tournament.

But if you allow yourself to update the book during rounds, you have to allow
your opponent to do the same. Else it is not realistic again. Even
engines/authors who have a little book ,that is much shorter but every move
checked, will react to what happens in the tournament games. E.g. Uri chose to
just switch books after watching a movei opening he didn't like in cct7.

The difference between a highly optimized book and one that just has few
adaptions is mostly in quantity in this discussion. While the optimized book
will usually have a few thousand manually entered lines the latter might have
only sth up to 50 ( numbers arbitrary chosen). The question is if you can be
sure that with the huge number of lines you don't add more garbage than quality
- a question that was easier to answer in the past than now, when you consider
engines that will typically think for 10 minutes for a first move out of book on
some monster hardware.

In one of my earlier experiments with chessprogram Bringer I once created a book
with hundreds of thousands of moves. It was very thorough. Unfortunately it
didn't pass the test to beat a book I had done in two hours that only had 20
nags :) .

I guess what comes closest to a reasonable test of such a tournament book is
running extremely short matches against a huge amount of engines/books. But to
get meaningful numbers you might need more opponents than availlable. I would be
interested to know how you did the experiment yourself.

Actually if the base of your book is anything created automatically, you have
the same basic problem as a completely automated book, sth unexpected might
happen.

Peter

This is not really organized, just some random thoughts. I think that your book
will win this contest, but I wouldn't be surprised if it was only by the closest
of margins.

Peter



This page took 0.02 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.