Author: Arturo Ochoa
Date: 11:03:38 02/15/05
Go up one level in this thread
On February 15, 2005 at 09:29:58, Peter Berger wrote: >On February 15, 2005 at 06:46:04, Arturo Ochoa wrote: > >>On February 15, 2005 at 02:56:54, Peter Berger wrote: >> >>>On February 14, 2005 at 20:08:42, Arturo Ochoa wrote: >>> >>>>On February 14, 2005 at 19:54:03, Peter Berger wrote: >>>> >>>>>Yes, you opposed this point of view multiple times before in discussions >>>>>with Uri , but I think you never managed to score. >>> >>>>Of, I have managed to score several games during the year 2004. >>> >>>That's just a misunderstanding, because I worded badly :). My "scoring" only >>>applied to the discussions, not to the quality of your work on the opening book. >>> >>>Btw - it's not trivial to think of a good and practical experiment to setup to >>>show who is right. >>> >>>Peter >> >>Well, I have already done such an experiment and I know who will give the >>easiest point. It is great because it will mean easy points. >> >>However, I would not spend again valuable time repeating the same nonsense >>experiment. >> >>Arturo. > >I think you can't really do such a test properly on your own. And it would be so >extremely time-consuming to do in a realistic way, that I doubt anyone has ever >really done it. > >You mentioned the major problem I see in another post, learning as done by a >book author. This is a factor that has to be taken into account. > >If you take your opening book as prepared for some major event, it is probably >nearly 100% deterministic at start of some round. If you run it against an >automatically learning book in longer matches to get a measure for the quality >of your work, it will get beaten badly. It might do well in the first few games, >until the opponent finds some hole ( which in this case means just some line >where it can beat "your" engine) - then it will repeat it in the following >games. How do you know that a book mine is so "deterministic"? How do you base your facts in more suppositions without any proof? Well, I can argue that I have done such tests but I wont reveal how I do such tests. > >This is not a realistic test of what would happen in a tournament. > >But if you allow yourself to update the book during rounds, you have to allow >your opponent to do the same. Else it is not realistic again. Even >engines/authors who have a little book ,that is much shorter but every move >checked, will react to what happens in the tournament games. E.g. Uri chose to >just switch books after watching a movei opening he didn't like in cct7. > >The difference between a highly optimized book and one that just has few >adaptions is mostly in quantity in this discussion. While the optimized book >will usually have a few thousand manually entered lines the latter might have >only sth up to 50 ( numbers arbitrary chosen). The question is if you can be >sure that with the huge number of lines you don't add more garbage than quality Well, As you said "usually" and your book was one thousand lines book. My book is bigger than only 1000 lines. This is not a relevant question because I dont generate a random book. My book has over 200000 lines added by hand. Sometimes, my decision about line has not been convenient. Maybe, you choose arbitraly your numbes. I dont know how you od it and I am not interested in. My books are higly checked by hands. Sometimes, I dont have enough time to test them with the engines and it possibly means a bad result. But your question is not relevant of how I do my books. >- a question that was easier to answer in the past than now, when you consider >engines that will typically think for 10 minutes for a first move out of book on >some monster hardware. > >In one of my earlier experiments with chessprogram Bringer I once created a book >with hundreds of thousands of moves. It was very thorough. Unfortunately it >didn't pass the test to beat a book I had done in two hours that only had 20 >nags :) . > Ah, now, your own tests says that a garbage book is not better than you fine book tuned by hand. Intresting. Your posting seems to be more than a groups of ideas that doesnt provide a good direction about what you try to mean. If this is a brain stormiong, let me know. I can also fill this page with a lot of irrelevant arguments. >I guess what comes closest to a reasonable test of such a tournament book is >running extremely short matches against a huge amount of engines/books. But to >get meaningful numbers you might need more opponents than availlable. I would be >interested to know how you did the experiment yourself. > No, I have worked hardly in some books over the last four years and I wont reveal how I do my testing or how I work. Why should I give important information for free in this Forum? Why? >Actually if the base of your book is anything created automatically, you have >the same basic problem as a completely automated book, sth unexpected might >happen. > >Peter My books is all tuned by hand. Sometime, I make a wrong decision about a line that I have not tested very well with the engine and a disaster can happen. > >This is not really organized, just some random thoughts. I think that your book >will win this contest, but I wouldn't be surprised if it was only by the closest >of margins. > >Peter Yes, it is not easy for me to follow your ideas. I would like a debate where oue ideas were clear and we avoided a waste of time, replying and replying. But, it is your option not mine. :)
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.