Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: CEGT: testing and presentation of results

Author: David H. McClain

Date: 03:23:34 10/20/05

Go up one level in this thread


On October 20, 2005 at 00:04:43, Heinz van Kempen wrote:

>Hi all :-),
>
>currently a few CEGT testers are disappointed about what was written here in
>some threads.
>
>Feeling not responsible for statistically absurd fluctuations we really think
>that it is better to give rating lists from Thursday evening onwards only for
>engine versions, settings, etc. with more than 300 games. It simply leads to
>disappointment and frustration, when there is a fantastic start and afterwards a
>cruel drop, like it happens so many times over and over again. Maybe better to
>inform the authors about progress of a big test via email or in our private
>forum.
>
>Joachim Rang wrote an email to CEGT testers that the chance to improve something
>by a setting might be only 1%. I think this is especially the case when few
>positions were tested only. Most of the good Blitz results were with additional
>king safety to 104, what we did not test.
>
>Moreover it can“t be that a whole team of experienced testers gets scrutinized,
>almost insinuating that there are essential things not done correctly. Of course
>with so many games it will not be possible to check the many bugs (every engine
>and GUI has) at once and theoretically there might be also a hardware or OS
>malfunction.
>
>We saw of course that all those discussions cast a negative light on our work.
>Feeling that we have a great team with enthusiastic testers currently even this
>will not demoralize us and we will try to offer valuable information to authors
>and interested readers in CCC also in the future.
>
>I regret that the discussion will surely have been annoying to the readers and
>will try to avoid unkind answers of any kind from my side, seeing that there are
>great efforts to make this forum friendlier overall.
>
>Best Regards
>Heinz
>
>http://www.husvankempen.de/nunn/

Heinz,

I think most of us know you and your testers are competent people.  Significant
fluctuations between tournaments involving the same engines between testers is
accepted and expected.

Please continue to post the results of your tests as you receive them.  I have
always accepted them at face value whatever they are.  DHM



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.