Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: test suites solicitation

Author: Uri Blass

Date: 06:37:52 01/22/06

Go up one level in this thread


On January 22, 2006 at 08:47:58, Tord Romstad wrote:

>On January 21, 2006 at 09:57:04, Jon Dart wrote:
>
>>I have built up a test suite from positions from
>>Arasan's games (plus a few from other sources).
>>Most of these have been verified by running them at
>>long time controls. They are probably comparable in
>>overall difficulty to ECMGCP.
>
>No.  ECMGCP is relatively easy on modern hardware, but
>your test suite seems to be very hard.  At 10 seconds per
>move on a Pentium IV 2.4 GHz, the latest version of my
>engine scores 159/183 at ECMGCP, but only about 50%
>in the Arasan suite (I don't remember the exact score).
>
>Tord

I think that for some reason your program is relatively better in ECMGCP and not
in arasan test suite but I also agree that probably scoring half of ECMGCP is
easier than scoring half of arasan for most programs.

Note that the difference is not very big and I think that if you take the easy
half of ECMGCP out of the test then the tests are going to be of similiar
difficulty(glaurung may still be better in ECMGCP but part of the other programs
will score better in arasan).

I think that scoring 50% of arasan and scoring 75% of ecmgcp are probably
problems of similiar difficulty.

Uri



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.