Author: Dann Corbit
Date: 09:04:47 02/01/06
Go up one level in this thread
On February 01, 2006 at 11:14:36, David B Weller wrote: >I was just here trying to figure out why my engine doesnt get a certain bm for a >positional test, and it occurred to me ... > >Why would I trust that? > >Many of the basic terms, eg., isolated pawn, have a fairly well established >value, representing a statisitical average over many, many positions > >If my engine,is missing some positonal move, for no other reason than I can >tell, except perhaps my isolated = 20 should be isolated = 25, then I am >disregarding the trillions of other positions where it is, statistically >speaking, really 20 > >As it has been pointed out many times, these tests suites are good only for >detecting gross errors > >So if you plan on tweaking the value of your SE metrics by test suites, make >sure it has about a million positions ;-) > >Maybe this is why 'auto' tuning is hard. Because if the suite doesnt contain >enough data to be representative of all the features one is trying to tune, it >will just be a waste of time, and make it worse... > >It could be that many problems can be easily solved, simply by inflating or >deflating the right term(s). And certainly a 'genetic' algorithm would find the >right ones to inflate/deflate on a small set of positions in order to get more >of them right... > >Fact is, it could be the very reason the position got in the test suite, is >because its is a little 'freakish'. Then what? We're tuning our engines to >become worse! > >my $0.02 > >IMHO > >-David And yet the really good engines tend to solve all of them, or nearly all of them. Of course, an equal problem to test suites is that all of them are full of outright mistakes and errors. Probably the best debugged suite is WAC and yet I imagine that it still contains errors. Good tactical ability is a necessary (but not sufficient) component for engine strength.
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.