Author: Robert Henry Durrett
Date: 07:42:46 06/22/02
Go up one level in this thread
On June 22, 2002 at 06:45:54, Mike S. wrote: >On June 21, 2002 at 18:11:21, Robert Henry Durrett wrote: > >>Where do the very best test suites and test positions come from? >> >>How are they created or selected? >> >>What distinguishes a really good one from the rest? > >As Dann already mentioned, many come from chess books. These are not the best >sources IMO, because the books are aimed at human use (and at least the older >one's have not been checked by computers, and of course not for suitability for >computer testing). > >When selecting good test positions, one major problem (or quality criteria) is >the "testing character" of the solution move. For example, if you take positions >which were selected for human training or reading pleasure so to speak, nasty >things like in-between moves (which may be unimportant for the human when they >are obviously waste of time), can ruin the test positions. Interested in better understanding of "nastiness" of in-between moves, as it applies to computer test suites. What "nasty" problems are caused by this? >Also, I dislike >"normal" solution moves (i.e. which sacs nothing, offers no sac...) because >you'll never be sure if the program chooses it for the right reason. This is >especially a problem, if you try to find good positional tests (which I quit >:o). Also, it's a problem when the 2nd best (which may actually be even the >best) move is very narrow in terms of evaluation to the best. Dann is an expert >in finding such problems in test suites (he has helped me with my previous test >suite). > >A good test position has > >1. perfect testing character of solution move (won't be played unless the >correct idea is found), most often achieved by sacrifices. So you don't have to >study "why has the engine played like that, what has it seen or not yet seen, >etc. etc." This is not useful for test positions IMO. >2. is neither too easy nor too difficult (this estimation changes with software >quality and hardware speed of course) >3. should be documented with source, and solution variant(s) >4. should offer a reasonable method to calculate and compare a total result > >Of course: The more postions like that, the better. If there are many positions >of a suitable diffculty (in average), counting and comparing the number of >solutions may be sufficient. With few positions, the solving time becomes >importand for comparisons. > >I have tried to meet the conditions above in my Quicktest. It isn't too >difficult, and is aimed at testing the analysis capabilities at *short* time >controls. I.e. if you quickly browse through your last 3.000 chess server games >:o), with only a few seconds per position. Which engine(s) are fastest in >finding the usual blunders & petit combinaisons? > >http://meineseite.i-one.at/PermanentBrain/quick/quicke.htm (description & >downloads) >http://meineseite.i-one.at/PermanentBrain/quick/quick3.htm (results P3/700) > >I think none of these positions will be solved for the wrong reason (pls note: >there are 2 avoid move tests among them). I've collected them from various >different sources, human and computer games, studies too. There is a bandwidth >of difficulty, from ~medium to easy. I don't think a program can solve all in >the intended 1 minute per position (but maybe 20 or 21/24 on very fast >computers). > >Regards, >M.Scheidl One idea which intrigues me that it may be possible to write a program which would search a large database of games to find positions which meet the criteria you suggest. Ever heard of anything like that? Think it is a good idea? I realize that the only large databases would be of human games, but maybe that's unavoidable? Bob D.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.