Author: Bruce Moreland
Date: 08:57:36 06/09/98
Go up one level in this thread
On June 08, 1998 at 21:45:43, Nobuhiro Yoshimura wrote: >When you modify your search engine or evaluation functions and test >aginst >many test suites, can you tell me how to decide whether you are going >to >adpot or reject the modification. > >For example: > before) 5sec 90% correct and 10sec 95%correct > after) 5sec 85% correct and 10sec 98%correct I haven't had to make this decision. If I had this situation, I'd look at things more carefully to try to figure out why. I'd also run other suites. 90% is 270, 95% is 285. 85% is 255, 98% is 294. 255 is especially bad, but the difference between 294 and 285 is also significant, although that'd be a place to look, to see if those extra nine correct answers were gotten for the wrong reasons. It would also be interesting to see how long it takes the first one to get 294. I would be inclined toward the second one, because hardware can only get faster, but it might be interesting to see if the initial zip in the first one can be incorporated into the second one. You don't usually see such a dramatic difference, and when you do, there isn't usually a tradeoff. One will usually be obviously a little better than the other. bruce
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.