Author: Kurt Utzinger
Date: 03:30:47 02/15/06
Go up one level in this thread
On February 15, 2006 at 05:28:36, Tord Romstad wrote:
>On February 15, 2006 at 02:34:28, Jouni Uski wrote:
>
>>It ruins my great test suites! It's very good in almost all testsuites tactical
>>and even endgame suites. In ECM based suite it's the best.
>
>Not quite. Hiarcs is definitely much better at the ECM. There could be other
>programs which perform better, too. I don't know.
>
>>But in real playing it's not so good. Why? Positional play??
>
>There are several reasons, and positional play is certainly one of them.
>Compared to the top programs, my evaluation function is quite poorly designed,
>and not very well tuned. At the moment, the program is also optimised for fun
>rather than for playing strength.
>
>The main weakness, however - and this may come as a surprise to you - is
>tactics. Most of Glaurung's losses against programs of similar strength
>are caused by tactical mistakes. The number of solutions found in tactical
>test suites have almost nothing to do with tactical abilities in real games.
>Tactical test suites have too many of the flashy types of tactics which
>human players find attractive, with lots of sacrifices and surprising and
>counter-intuitive moves. These are hardly representative for the tactics
>which tend to decide computer vs computer games. Most of the important
>tactics don't involve sacrifices at all.
>
>Another important factor to be aware of is that it is the worst case
>behavior of the program which is most important for the practical strength.
>Imagine a program which solves 99% of the most difficult tactical test
>positions instantly, but makes elementary tactical blunders in 1% of the
>positions which occur on the board when playing games. It would outperform
>any other program in test suites, but would be tactically outplayed all the
>time when playing against other engines.
>
>That Glaurung performs so well at many tactical test suites is a great
>mystery, by the way. It is a rather slow program, and it doesn't extend
>much. I suspect that the many fast solutions are caused by some subtle
>search bug.
>
>Tord
Hi Tord
Very interesting (and convincing) explanations. You have correctly
stated that most tactical test suites "measure" the wrong things,
in any case not what is important for the practical playing strength
of an engine.
Best regards
Kurt
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.