Author: Robert Hyatt
Date: 15:32:35 06/01/04
Go up one level in this thread
On June 01, 2004 at 18:09:11, Sune Fischer wrote: >On June 01, 2004 at 14:01:57, Robert Hyatt wrote: > >>On June 01, 2004 at 12:27:44, Sune Fischer wrote: >> >>>On June 01, 2004 at 11:56:57, Robert Hyatt wrote: >>> >>>>> >>>>>It's a very powerful feature, too powerful IMO if not all engines have it. >>>>>I'm quite sure even Ruffian would lose 10-90 if Crafty had aggressive learning >>>>>and Ruffian just used a small book without learning. >>>>>You can be of the opinion that's a fair result, I think it is pure nonsense. >>>>>Granted, it demonstrates that Crafty has learning that works, but what other >>>>>conclusions can you hope to draw from it? >>>> >>>>Reverse the argument. Crafty vs Fritz. No learning. But Fritz has a book that >>>>has been hand-tuned by a full-time human working on the book. Is _that_ a fair >>>>test then. In one case an engine learns by itself and it is turned off. In the >>>>other case, the engine learns by a human hand-tuning the book, but _that_ isn't >>>>turned off. >>>> >>>>Perhaps I miss the point, therefore? It isn't balanced either way... >>>> >>>>At least with learning, the hand-tuned book won't win the _same_ opening >>>>repeatedly... >>> >>>That's true, but I don't think it's Fritz's job to educate Crafty. >> >>And _there_ we disagree. I don't have a full-time book person. So to you it is >> fair that "the bigger the team, the more advantages the program gets?" I >>personally think it is _everybody's_ job to educate Crafty. And by golly, I >>wrote code to make that happen. :) If you turn off my learning, turn off the >>customized books _they_ use. Otherwise it is just a random and unfair testing >>methodology... > >If the books are that terrible, why not pre-learn them a bit instead of shipping >them raw? Against what opponent? How can I know _before_ the fact? IE if I train the book on ICC, that won't be relevant to a hand-prepared and non-public book, such as those used in the SSDF. I don't see how to solve this in any way other than to run it "as written" and let it learn what it can and hopefully after taking a few early lumps it will get into reasonable book lines. > >>>When you turn learning off you test the two systems "as is". >> >>rat snot. :) >> >>Crafty "as is" is built _around_ book learning. So you most definitely are >>_not_ testing it "as is"... >> >> >> >> >>>Ie. how strong is Crafty for those who go to your ftp and download it _right >>>now_, compared to how strong is Fritz if you go and buy it, right now. >> >>Ask the right question. Is Crafty's default book, made 100% automatically, as >>good as a hand-tuned book with years of effort? Of course it isn't. But can >>its book become as good as Fritz's? Of course it can, just let them play. Then >>the game evolves into engine vs engine, which is what everyone really thinks >>they are measuring when they are really measuring anything but that. > >I don't think Crafty's book with perhaps a million moves is going to turn into a >super book because you play 10, 20 or even 100 games. Of course it isn't. But it is also not going to stay a "cannon-fodder" book either, as it will quickly veer away from prepared lines that cause real trouble... > >All the mistakes in the book still have to be made once before they are removed, >and that's assuming the learning works perfectly. Not quite. results get propagated backward through the book after a single game, possibly even changing the next game's first move... And then the results inter-mingle quickly. This isn't a dumb "here was the last choice, don't play that move again..." sort of deal. It is more sophisticated than that by a bunch... > >Cleaning a book like that is likely to take years, hardly something you can >expect of the average user. Remember that you learn good and bad things. If the opponent plays a line that you don't like, chances are you will like it from his side. This happens quickly... > >>>I think that is a valid question and perhaps more relevant than asking how big >>>Crafty's potential is if the user trains it for 500 games. >>> >> >> >>I think the question is just as pointless as pondering what is 2+2 for months. >>The answer is already known. A hand-tuned book is better. I simply chose to >>try to develop a solution to make that work unnecessary. You choose to >>eliminate the solution and introduce a much larger error into the comparison... > >Crafty is not the only engine with learning. Of course not. Didn't say that it was, I hope. > >I don't see why it should be unfair to turn it off, as long as you turn it off >for everybody. Again, suppose some don't have it at all. They lose _nothing_. Suppose they have a hand-crafted book. They still will avoid bad lines. The most level playing field there is is to play each program "as is" and leave them alone to do what they want, when they want to do it, and how they want to do it... > >-S.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.