Author: Ingo Lindam
Date: 01:56:42 12/09/02
Go up one level in this thread
On December 09, 2002 at 01:22:51, Leen Ammeraal wrote: >On December 09, 2002 at 00:11:41, Mike S. wrote: > >> >>(c) The first move of the solution should have a "testing character". This >>means: The first move should be unusual (would commonly bad), for that it's only >>chosen when the engine has seen the idea behind the move. Sacrifices are easy >>examples for that. >> >>If the solution starts with a "normal" move, which sacs nothing or refutes no >>sac etc., the danger is big that it's just chosen by luck. You cannot easily >>tell if the engine has seen the idea in such cases. >> > >But might such a testsuite not give an unrealistic outcome >with regard to a program's strength? I am afraid you are right, that atleast it gives the chance to get an unrealistic outcome. I would say every unrealistic set of testpositions includes this chance of unrealsitic outcomes. Also to include only positions where the best move is much more better than the second best is not a realistic setting. The most realistic set of test positions in my eyes sould be to randomly choose a very large number of positions out of real life chess (positions occurred in games, this should be atleast a 'good choice' as long as chess engines don't have whole chess history in their brains) BUT on the other hand you ofcourse need a testsuite that you are able to evaluate the test results for (if possible in a reasonable time). And I guess this is the much more difficult point in finding a good testsuite and evaluation method. However you choose the positions for you test set it keeps diffcult to test the idea or plan an engine obtains in a given position only by a single best move proposal. If you would like to evaluate ideas and plans then you have to ask the engines for ideas and plans instead of a single 'best move'. Unfortunately they will not answer on such a question nowadays. Internette Gruesse, Ingo >For example, I could make my engine try all captures that look bad >before all other legal moves. >That would give a good score with your testsuite, because such >bad captures are likely to be sacrifices. But in realistic play, >such sacrifices do not occur in most positions, so this strategy >of investigating apparent bad captures will give a bad result >in actual games. > >Leen
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.