Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Results of 112 engines in test suite "WM-Test" (100 pos) for download

Author: Mike S.

Date: 22:42:07 08/20/02

Go up one level in this thread


On August 20, 2002 at 17:25:43, Uri Blass wrote:

>(...)
>No reason to be surprised.
>This test was designed to put Fritz on top because the analysis was done by
>Fritz to decide about the positions.

What exactly do you mean?

Do you really mean, "designed to put Fritz on top"? If not - and I assume (and
hope) you didn't mean that - then I suggest to be a bit more careful. Now it
sounds like an accusation.

Btw. you probably missed that the test has been developed in several stages
(with increasing number of positions), where in between *Gromit* 3.9.5 (!) once
was on top. The first version (40 pos.) was designed and published before Fritz
was available.

>I am sure that Fritz is going to have bad result if someone design a test to put
>tiger on top.

It would be unethical, and I don't expect somebody would do that or ever has
done - if he would hide the intention. No matter which program is favoured of
course (comment for the suspicious :o).

OTOH, it could be interesting as an experiment if one would explain it before,
that he'd try to see if it is possible at all. It would require to choose
positions where not only Tiger is good, but also Fritz is slower at the same
time (or in the majority of the positions). I think while both are about equally
strong in gameplay, Tiger 14 is usually behind Fritz 7 in testsuites, in
general. Which doesn't mean it couldn't solve faster then and when, than Fritz 7
(especially in the endgame; CT 14 is *ahead of F7* in the endgame part of the WM
Test!).

I found that in the WM Test, Tiger 14.0 was better than Fritz 7.0.0.8 in 31
positions, Fritz 7.0.0.8 was better in 52, and the rest (17) both didn't solve.

Hm... it seems that with a *smaller selection* of large test suites, you could
"prove" anything (also with other pairs of programs within a reasonable
bandwidth). But I don't expect that anybody would ever do that. It would be
quite cruel to suspect that!

Even manufactureres - where it's normal to present good performances of their
programs only - don't do that. They usually compare with previous versions of
their own programs, and don't present such large test suites with results (which
would be interesting though, because there surely is interesting data, special
position collections etc. hidden in the test labs :o).

Regards,
M.Scheidl



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.