Author: Albert Silver
Date: 05:37:13 08/17/02
Go up one level in this thread
On August 17, 2002 at 05:11:01, Uri Blass wrote:
>On August 16, 2002 at 17:39:48, Manfred Meiler wrote:
>
><snipped>
>>For all the other mentioned engines it seems to me that this test suite is too
>>hard for them.
>>The WM-Test was designed by the test authors Gurevich/Schumacher for engines
>>with playing strength in a range of 2500 - 2800 ELO (look at the attached readme
>>file) - in this point of view (ELO >=2500) I shouldn't have tested many of these
>>112 engines... :-)
>
>Did they check carefully that the test is correct?
>
>When I look at the pgn it seems that lines are not convincing
>for computer programs.
>
>For example in position 3 they give the line
>1.Nf5 gxf5 2.gxf5 Nc7 3.Rg1 Ne8 and no word about the typical computer
>move Rf7.
>
>After Ne8 yace can see immediatly small advantage for white
>Bxh6 0.54/10,0.65/11,0.54/12
>but after Rf7 it says Rg6 -0.37/10,Rxg7+ -0.28/10,
>Rxg7+ -0.47/11,Rg6 -0.30/11,Rg2 -0.21/11
>Rg2 -0.22/12
>
>Maybe it can see advantage for white after long analysis
>(I did not try it) but I think that it is better to give some
>tree to convince programs that the moves are correct(At least in part
>of the cases I cannot use the pgn together with yace's learning to prove that
>the solutions are winning moves in a reasonable time and I often
>cannot even convince it that the move to find is the best move).
>
>Uri
I gave similar questions to one of the actual authors, Gurevich, in the
Chessbase forum, but with little effect. I showed him that several engines could
reach the correct move, and keep it, for reasons that had nothing to do with the
solution. For example, in one position IIRC, the key _winning_ move was Re3, yet
several engines chose it because they thought it led to a perpetual check and
considered the playing side to be slightly inferior. I thought that although one
can argue the move was played, that if it is to show an engine's competence in
finding _winnning_ moves, it should of course be clear that is why it played it.
I argued merely that in such a case, the position should not be a part of the
suite, since it wasn't clear the engine would actually win the game. There was
at least one other case where an engine chose the winning move for reasons that
had nothing whatsoever to do with the winning line, and the line the engine gave
showed it had no understanding as to why the move was great. In other words it
might play this wonderful move, a key to winning the position, but still not win
the game. Again, in my eyes this meant that it was poorly suited to be a part of
a test suite. I still thanked him for the wonderful effort. The author was very
courteous, but treated my arguments as clever rhetoric, and ignored them.
Albert
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.