Author: Jay Scott
Date: 19:00:04 02/25/98
Go up one level in this thread
On February 25, 1998 at 18:16:51, Amir Ban wrote: >On February 25, 1998 at 17:23:52, Jay Scott wrote: > > >>A chess program is in an exactly analogous position. It would >>like to be as discriminating as possible while staying well- >>calibrated. >> >>I don't think it's a serious problem in practice. You can >>measure everything from game data. >> > >I don't understand this part: Are you saying my best-fit method would >give good results in practice ? Or do you have a different procedure in >mind ? If you have an actual program with an actual evaluation function, you can collect data by playing games and find out whether the evaluator is well-calibrated (accurate but possibly vague) and whether it's what I've called discriminating (precise but possibly inaccurate; claiming to know more than it does, if you like). To me it seems pretty clear that in theory you can always adjust an evaluator to be well-calibrated, using the game data, and that it's not going to hurt to do that (at least not on average). So the idea is to make the evaluator as discriminating as possible without losing accuracy. Also, I think a statistically valid best-fit procedure will tend to do this automatically, as it were, so it seems like a good idea to me. One of the tricky parts is to fit to data that corresponds to the situations your program actually has to make decisions about--I think this may be harder than it seems. The popular idea of fitting to grandmaster games is probably bad for chess programs, for example, because they're so different from grandmasters. Jay
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.