Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Results of 112 engines in test suite "WM-Test" (100 pos) for download

Author: Vincent Diepeveen

Date: 11:49:55 08/17/02

Go up one level in this thread


On August 17, 2002 at 05:11:01, Uri Blass wrote:

>On August 16, 2002 at 17:39:48, Manfred Meiler wrote:
>
><snipped>
>>For all the other mentioned engines it seems to me that this test suite is too
>>hard for them.
>>The WM-Test was designed by the test authors Gurevich/Schumacher for engines
>>with playing strength in a range of 2500 - 2800 ELO (look at the attached readme
>>file) - in this point of view (ELO >=2500) I shouldn't have tested many of these
>>112 engines... :-)
>
>Did they check carefully that the test is correct?

No they didn't, they are too stubborn for that.
I gave also clear examples of where other moves are more
convincing, but the proof of the authors is a mainline which
fritz prints within 10 seconds. Then by exception they will
run it for 10 minutes on fritz7. If it isn't showing your
better move then, then that's their proof that it isn't so well.

Look these guys i couldn't find on any rating list. Perhaps one of
the guys has a 2170 rating from 20 years ago. That doesn't mean they
know anything from how to analyze a position.

Their basic 'analysis' is by relying upon the authors of the problems
and putting it to Fritz. A good example is the Ba5 move one of the
first problems. That some topgrandmaster like Anand preferred the
move doesn't mean it is *objectively* the best move. Who knows whether
he was in time trouble or simply wanted to avoid tactics?

The only proof i was mailed back was a mainline from fritz7. That's sick
simply. They are too bad in chess to analyze themselves even! All they
can analyze is by making 1 move with fritz and then another move with
fritz.

Obviously the lack of chessknowledge of the authors means that they are
not capable of putting simple positional problems into the testset where
nearly all engines go wrong. That's beyond their own knowledge.

I could make a testset with hundreds of positions from my own games, or
analysis from it, which shows all kind of positional problems. That would
take me at most 7 evenings of 1 week.

Of course i'm not going to do it. That's the only thing they are good in,
putting time in testing engines to the positions. Of course they are not
going to test movei or any other 'amateur' engine. Because just *suppose*
that one of the engines is very aggressive tuned and scores real high
on their testset.

How's CSTII doing on this testset, speaking of an aggressive but very
weak engine?

>When I look at the pgn it seems that lines are not  convincing
>for computer programs.
>
>For example in position 3 they give the line
>1.Nf5 gxf5 2.gxf5 Nc7 3.Rg1 Ne8 and no word about the typical computer
>move Rf7.
>
>After Ne8 yace can see immediatly small advantage for white
>Bxh6 0.54/10,0.65/11,0.54/12
>but after Rf7 it says Rg6 -0.37/10,Rxg7+ -0.28/10,
>Rxg7+ -0.47/11,Rg6 -0.30/11,Rg2 -0.21/11
>Rg2 -0.22/12
>
>Maybe it can see advantage for white after long analysis
>(I did not try it) but I think that it is better to give some
>tree to convince programs that the moves are correct(At least in part
>of the cases I cannot use the pgn together with yace's learning to prove that
>the solutions are winning moves in a reasonable time and I often
>cannot even convince it that the move to find is the best move).
>
>Uri



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.