Author: Peter Berger
Date: 06:29:58 02/15/05
Go up one level in this thread
On February 15, 2005 at 06:46:04, Arturo Ochoa wrote: >On February 15, 2005 at 02:56:54, Peter Berger wrote: > >>On February 14, 2005 at 20:08:42, Arturo Ochoa wrote: >> >>>On February 14, 2005 at 19:54:03, Peter Berger wrote: >>> >>>>Yes, you opposed this point of view multiple times before in discussions >>>>with Uri , but I think you never managed to score. >> >>>Of, I have managed to score several games during the year 2004. >> >>That's just a misunderstanding, because I worded badly :). My "scoring" only >>applied to the discussions, not to the quality of your work on the opening book. >> >>Btw - it's not trivial to think of a good and practical experiment to setup to >>show who is right. >> >>Peter > >Well, I have already done such an experiment and I know who will give the >easiest point. It is great because it will mean easy points. > >However, I would not spend again valuable time repeating the same nonsense >experiment. > >Arturo. I think you can't really do such a test properly on your own. And it would be so extremely time-consuming to do in a realistic way, that I doubt anyone has ever really done it. You mentioned the major problem I see in another post, learning as done by a book author. This is a factor that has to be taken into account. If you take your opening book as prepared for some major event, it is probably nearly 100% deterministic at start of some round. If you run it against an automatically learning book in longer matches to get a measure for the quality of your work, it will get beaten badly. It might do well in the first few games, until the opponent finds some hole ( which in this case means just some line where it can beat "your" engine) - then it will repeat it in the following games. This is not a realistic test of what would happen in a tournament. But if you allow yourself to update the book during rounds, you have to allow your opponent to do the same. Else it is not realistic again. Even engines/authors who have a little book ,that is much shorter but every move checked, will react to what happens in the tournament games. E.g. Uri chose to just switch books after watching a movei opening he didn't like in cct7. The difference between a highly optimized book and one that just has few adaptions is mostly in quantity in this discussion. While the optimized book will usually have a few thousand manually entered lines the latter might have only sth up to 50 ( numbers arbitrary chosen). The question is if you can be sure that with the huge number of lines you don't add more garbage than quality - a question that was easier to answer in the past than now, when you consider engines that will typically think for 10 minutes for a first move out of book on some monster hardware. In one of my earlier experiments with chessprogram Bringer I once created a book with hundreds of thousands of moves. It was very thorough. Unfortunately it didn't pass the test to beat a book I had done in two hours that only had 20 nags :) . I guess what comes closest to a reasonable test of such a tournament book is running extremely short matches against a huge amount of engines/books. But to get meaningful numbers you might need more opponents than availlable. I would be interested to know how you did the experiment yourself. Actually if the base of your book is anything created automatically, you have the same basic problem as a completely automated book, sth unexpected might happen. Peter This is not really organized, just some random thoughts. I think that your book will win this contest, but I wouldn't be surprised if it was only by the closest of margins. Peter
This page took 0.04 seconds to execute
Last modified: Thu, 07 Jul 11 08:48:38 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.