Author: Sune Fischer
Date: 11:26:05 06/02/04
Go up one level in this thread
On June 02, 2004 at 13:40:28, Robert Hyatt wrote: >On June 02, 2004 at 12:16:16, Sune Fischer wrote: > >>On June 02, 2004 at 11:42:24, Robert Hyatt wrote: >> >>> >>>> >>>>> But even >>>>>then, you have _real_ problem because there is some randomness built into my >>>>>move selection logic to provide variety. >>>> >>>>That's annoying yes, but as long as it averages the same strength it might not >>>>be totally damaging. >>> >>> >>>Playing the Sicilian in one match as black, the Latvian in the next match will >>>not "average the same strength"... >>> >> >>Doesn't matter, what matters is playing the sicilian and Latvian at constant >>levels. > >Do you play 100,000 game matches? If not they will _not_ be at "constant >levels". My book selection has some randomness built in that will take it into >oddball (and unsound) lines with some frequency... > > >> >>Eg. suppose that in one of our basement tournaments Crafty gets the same >>sicilian twice against two engines of equal strength. >>Crafty falls into a trap in the first game, "learns" and manages to avoid it for >>the next match. This will only punish the latter of the two opponents. > > >Isn't that _exactly_ how a human works? Invite me to a "basement tournament" >but tell me I can't learn from one game to the next and see what happens. :) Your want Crafty to look good at all cost, that's a fair point. Others are interested in the level of the analysis engine without all the shinanigans, that's a fair point too. I see no conflict, there should be room for both. >>I think that is unacceptable in a testing environment. > >You are seriously mixing terms. Are you talking about a basement tournament, >where a third-party (not the programmer of any engine) or a basement match (same >idea), or are you talking about _you_ testing your engine (as a programmer) >against mine? I'm not mixing anything, I'm saying there should be room for everything. You intended for your engine to play with x,y,z settings, another author intended for his engine to play with q,r,s settings. Unfortunately x,y,z conflicts with q,r,s so you have to make a compromise somewhere. >If the latter, then why are you in this conversation? I _specifically_ >addressed a "basement tournament by a non-programmer with learning disabled." I >have been _specifically_ addressing that concept from the beginning. > >So let's fix the setting first, and stay on the same page together. I'm talking >about "third-party events" _only_... Third-party events have all sorts of agendas, some are interested in testing under those conditions others are interested in something else. It's not for you to decide how people use Crafty. You may say "if you do this or that and then you weaken Crafty", fair enough, now we know. >> >>>>>If you play a 20 game match, make >>>>>changes, and play another 20 game match, comparing the results is less than >>>>>worthless... >>>> >>>>So maybe Crafty is just worthless for testing, that is possible. >>> >>>Or perhaps your testing methodology is worthless... Crafty is not that >>>different from any other program. You have to be sure to play enough games to >>>hide the random factor. >> >>I can't do that against Crafty, if it learns it's a moving target. >> > >And if it doesn't learn it is a random target. If you insist on controlling people 100% well maybe the best solution for you is to not give Crafty away. I know you always want the last word on everything so I think I'll end just stop here, my point has been made clear, to most people, I think. -S.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.