Author: Dann Corbit
Date: 18:13:49 04/27/04
Go up one level in this thread
On April 27, 2004 at 21:07:52, Eric Oldre wrote: >On April 27, 2004 at 18:06:17, Dann Corbit wrote: > >>On April 27, 2004 at 17:57:15, Eric Oldre wrote: >> >>>My engine (murderhole) has gotten to the point where it's not easy for me to >>>tell if a given change has helped or hurt more. So I need to come up with a >>>better, more quantitative way of testing. >>> >>>Of course I know that the only sure test is lots and lots of games. but i just >>>don't have the patience, and the results can vary so much. >>> >>>my idea, and i'm sure you all have thought of something similar, but probably >>>better (that's why i'm posting) is: >>> >>>1) create a series of test positions probably small at first (50-100) but would >>>need to grow later. >>> >>>2) generate a list of all possible moves from each test position and the >>>resulting position after the move. >>> >>>3) have some strong program generate scores for each resulting position, and >>>therefore a score for the preceeding move. >>> >>>4) then i could run my program against each position and see how often it picked >>>the best, 2nd best, 3rd best, etc. >>> >>>I could store all the test positions and scores in an XML file perhaps. >>> >>>The only problem is that setting this up would be a pain and spare time is not >>>something I have lots of these days (like all of us i'm sure) so if i can avoid >>>some work i'm all for it.. I was hoping someone might have some files for >>>configuring this stuff publicly available, or maybe it is even a feature of some >>>commercial program that i don't know of. >>> >>>Even if only pieces of this are out there it could help. Or something similar. >>> >>>Any ideas? >> >>Test suites do not work for this purpose. They are good for judging tactical >>strength, but poor estimators of game strength. If you optimze for tactics, >>then the program will play poorly. I can generate a large boost in the tactical >>strength of Beowulf by tuning using test suites. Then it gets murdered in >>actual games. >> >>It may be that quiet moves could be a good indicator. >> >>Another possibility is to look at what Dave Gomboc did in his thesis. >>Seems like it would require lots of hardware, though. > >Dann, >I'm not sure I fully understand. If the program improves tactically, then what >would make it weaker in game situations? Good tactics are a necessary, but not sufficient need for a strong chess program. You will find that tactical shots often involve sacrifices and other risky behaviors. So if you tune to tactical suites, it will cause problems. >At least at the stage my engine is at, there is still so much room for >improvement that there must be room to improve tactically without hurting other >aspects. You can automatically tune them if you like. At least it will give you a good tactical strength setting. I use a parabolic fit, under the assumption that taking the parameter to extreme values will be detrimental and some central value will be good (e.g. if a pawn is 100, then a value for the knight between 2 and 4 pawns will be much better than 0 pawns or 8 pawns.)
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.