Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Usage of general.ctg book+CB learner by ChessMaster in SSDF testing

Author: Mogens Larsen

Date: 01:29:57 10/31/03

Go up one level in this thread


On October 31, 2003 at 04:15:26, Luis Smith wrote:

>Its not a matter of common sense, its a matter of opinion.

No, it isn't. The general requirement of testing the actual product and not
permutations doesn't allow for opinion. No matter how well founded.

>Mogens:  I suspect the general consensus is that testing conditions should be a
>matter
>between author and SSDF.
>
>Do you agree with this consensus?  Besides why should the author get a say-so?
>SSDF owns the programs.

My "requirement" is out of the box. There are minor parameters, where it makes
sense to ask the author, eg. hashsize, tablebases, book (if more than one), GUI
(if more than one), etc. But no tampering with the engine unless it's generic.

>And secondly, that testing should be conducted without
>>advantages or disadvantages (within reason), or not at all.
>
>Doesn't matter its SSDF's choice.

Yes, it matters. An unfair ratinglist isn't worthwhile and doesn't distinguish
it from your average basement tournament. In one word. Credibility.

Regards,
Mogens



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.