Author: Kurt Utzinger
Date: 22:23:10 10/19/05
Go up one level in this thread
On October 20, 2005 at 00:04:43, Heinz van Kempen wrote:
>Hi all :-),
>
>currently a few CEGT testers are disappointed about what was written here in
>some threads.
I fully understand them. Most people who do criticise
have never thought about what real testing means :-)
Kurt
>
>Feeling not responsible for statistically absurd fluctuations we really think
>that it is better to give rating lists from Thursday evening onwards only for
>engine versions, settings, etc. with more than 300 games. It simply leads to
>disappointment and frustration, when there is a fantastic start and afterwards a
>cruel drop, like it happens so many times over and over again. Maybe better to
>inform the authors about progress of a big test via email or in our private
>forum.
A good idea. It's simply true that no conclusions whatsoever
should be made about an engine before 300-500 games have been
played. Those comments based on 10 games only are absolutely
irrelevant in my opinion. The chess computer community should
by now know this fact.
Kurt
>
>Joachim Rang wrote an email to CEGT testers that the chance to improve something
>by a setting might be only 1%. I think this is especially the case when few
>positions were tested only. Most of the good Blitz results were with additional
>king safety to 104, what we did not test.
>
>Moreover it can´t be that a whole team of experienced testers gets scrutinized,
>almost insinuating that there are essential things not done correctly. Of course
>with so many games it will not be possible to check the many bugs (every engine
>and GUI has) at once and theoretically there might be also a hardware or OS
>malfunction.
>
>We saw of course that all those discussions cast a negative light on our work.
>Feeling that we have a great team with enthusiastic testers currently even this
>will not demoralize us and we will try to offer valuable information to authors
>and interested readers in CCC also in the future.
>
>I regret that the discussion will surely have been annoying to the readers and
>will try to avoid unkind answers of any kind from my side, seeing that there are
>great efforts to make this forum friendlier overall.
You all are doing a great job. Please continue and simply
do not read/hear the nonsense some people are writing in
this forum.
Kurt
>
>Best Regards
>Heinz
>
>http://www.husvankempen.de/nunn/
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.