Author: Don Dailey
Date: 13:55:48 09/06/98
Go up one level in this thread
>I don't think SELF test (x vs. x+1) makes much sense. Sorry. >My method is different. > >I watch a game, or more games, and try to find out subjective way, using my >feelings and emotions. And main-lines. >And i do not bean count nor use statistics. > I think your method is pretty flawed if it depends very much on subjective analysis. For instance, if I did not like a program, I would be looking for problems and I would find them. I would dismiss the good moves and make excuses. I wouldn't do this on purpose, I would do it simply because I was human. Your method should always be a part of the whole testing philosophy however. You can get into the trap of playing thousands of games and not seeing a single one. I think you must continue to do what you do, but other more objective test MUST also be performed. And taking a liking to the way a program plays is just a whole lot different from knowing if it can get results. Self testing is an important part of the way we test. It is ideal for some things, not very good for others. We test all sorts of different ways and watch our program play to learn what is wrong. I don't think anyone has found the idea method for accurately measuring tiny program improvements. If you give me a way to measure a 1 rating point improvement, I will write you a chess program that beats all the others with no question about who is strongest. Larry had a theory that self testing exaggerates improvements. If you make a small improvement, it will show up more with self testing. If this is true (we called it a theory, not a fact) then this is a wonderful way to test because small improvements are so hard to measure and this exaggerates them. We know that self testing won't expose our program to weaknesses that it itself cannot exploit. That's why we consider it as a supplement to other kinds of testing. - Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.