Author: Mike S.
Date: 03:45:54 06/22/02
Go up one level in this thread
On June 21, 2002 at 18:11:21, Robert Henry Durrett wrote: >Where do the very best test suites and test positions come from? > >How are they created or selected? > >What distinguishes a really good one from the rest? As Dann already mentioned, many come from chess books. These are not the best sources IMO, because the books are aimed at human use (and at least the older one's have not been checked by computers, and of course not for suitability for computer testing). When selecting good test positions, one major problem (or quality criteria) is the "testing character" of the solution move. For example, if you take positions which were selected for human training or reading pleasure so to speak, nasty things like in-between moves (which may be unimportant for the human when they are obviously waste of time), can ruin the test positions. Also, I dislike "normal" solution moves (i.e. which sacs nothing, offers no sac...) because you'll never be sure if the program chooses it for the right reason. This is especially a problem, if you try to find good positional tests (which I quit :o). Also, it's a problem when the 2nd best (which may actually be even the best) move is very narrow in terms of evaluation to the best. Dann is an expert in finding such problems in test suites (he has helped me with my previous test suite). A good test position has 1. perfect testing character of solution move (won't be played unless the correct idea is found), most often achieved by sacrifices. So you don't have to study "why has the engine played like that, what has it seen or not yet seen, etc. etc." This is not useful for test positions IMO. 2. is neither too easy nor too difficult (this estimation changes with software quality and hardware speed of course) 3. should be documented with source, and solution variant(s) 4. should offer a reasonable method to calculate and compare a total result Of course: The more postions like that, the better. If there are many positions of a suitable diffculty (in average), counting and comparing the number of solutions may be sufficient. With few positions, the solving time becomes importand for comparisons. I have tried to meet the conditions above in my Quicktest. It isn't too difficult, and is aimed at testing the analysis capabilities at *short* time controls. I.e. if you quickly browse through your last 3.000 chess server games :o), with only a few seconds per position. Which engine(s) are fastest in finding the usual blunders & petit combinaisons? http://meineseite.i-one.at/PermanentBrain/quick/quicke.htm (description & downloads) http://meineseite.i-one.at/PermanentBrain/quick/quick3.htm (results P3/700) I think none of these positions will be solved for the wrong reason (pls note: there are 2 avoid move tests among them). I've collected them from various different sources, human and computer games, studies too. There is a bandwidth of difficulty, from ~medium to easy. I don't think a program can solve all in the intended 1 minute per position (but maybe 20 or 21/24 on very fast computers). Regards, M.Scheidl
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.