Author: Ed Schröder
Date: 01:00:31 04/28/98
Go up one level in this thread
>Posted by Thoralf Karlsson on April 27, 1998 at 15:32:56:
>In Reply to: Re: Tournaments posted by Ed Schröder on April 27, 1998 at
>11:26:53:
>On April 27, 1998 at 11:26:53, Ed Schröder wrote:
> >>If Fritz autoplayer is clean, does it increases the possibility of
>>>future Rebel coming into SSDF again? I hope so!
>>I don't think so. Rebel and auto232 has always been PUBLIC. Hidden
>>autoplayers have an advantage above the public ones. This is unfair
>>competition.
>Hi Ed,
>If I understand you correctly, you haven't so far found any signs of
>cheating when Fritz5's autoplayer is used against Rebel 9. If yours and
>Enriques further testing confirms this, I presume that the hypothesis of
>cheating can be put aside.
>You say that hidden autoplayers have an advantage above the public ones
>and that this is unfair competition. I agree that it costs more for you
>to play manually against Fritz 5 than for ChessBase to play
>automatically against your program. Still I cannot understand why it
>should be a task for SSDF to secure that the economical conditions for
>chess programming are equal. As far as I can see they are probably
>rather unequal, even if all autoplayers were public.
>You said in an email that it would have been OK for you if SSDF had
>played the Fritz 5-games manually. But assuming that there is no
>cheating, why would it be better if SSDF used an extremely slow (and by
>the way practically speaking impossible) way of confirming the rating of
>Fritz 5? The rating would still be the same apart from statistical
>variations.
>The original reason for the SSDF rating list was to give the consumers
>information about the relative strengths of chess computers/programs. A
>buyer of Fritz 5 wants, among other things, to know how good it plays
>against other computers. I don't think that they care if the production
>costs has been somewhat higher or lower.
>A question: If ChessBase would release their autoplayer, would then
>everything be OK for you?
>>Furthermore I regret the silence of the SSDF very much. 4 main chess
>>programmers have complained. The complaints are from end of February.
>>Now we are 2 months further. No official answer, just silence, silence
>>and silence.
>Some of the firsts complaints came from Ossi Weiner in a group email,
>which I responded. He didn't react on my answers but instead sent
>several new emails with fierce attacks on the SSDF rating list. Many
>other persons on CCC and in the group emails answered Weiners letters,
>so in my opinion it wasn't so urgent to answer them.
>Several persons, some known, some unknown have received answers to their
>private emails concerning Fritz 5. Some answers have also been given in
>group emails.
>I have answered three of your emails, although I admit it took
>unnecessarily long time. You know that I have sent emails to most of the
>programmers asking them about the possibilites for cheating with
>auto232, and other questions. This has been done in an honest effort to
>take your accusations seriously, even if I didn't believe in them. It
>takes time to investigate all of the hypotheses put forward. This is a
>hobby for me. I have several other things to do.
>I could have defended the SSDF rating list better, OK. But your
>description of the complete silence from SSDF isn't correct.
>>10-14 days ago SSDF posted that there will be no April list. Why?
>I simply don't have time for it at the moment. And before something is
>officially decided, matters have to be investigated first. This takes
>time.
>>Furthermore the posting also said, next list in May or June. I then
>>decided to quit SSDF. I don't like to wait another 2 months on an
>>answer, also I don't like to be ignored.
>I'm sorry if you feel ignored. How do you think I feel?
>Thoralf Karlsson
-----------------
Hi Thoralf,
Thanks for your comments, I like to make my position clear.
First, I have been in love with your list and your work since SSDF was
founded. What year was that? I believe it was 1984. In that 14 years
you have build yourself quite a reputation as the main (almost only)
source for testing commercial and other general available chess software
concerning playing strength by playing thousands of computer-computer
games.
You earned a lot of credit, like to quote my own home page now:
SSDF is founded in 1984 and has been since then the general
accepted computer rating list for chess magazines and computer
chess lovers all over the world. The SSDF received this credit
by their excellent work concerning honest an decent testing
methods.
In the past I have criticized your list, last time I did that was when
Rebel 8.0 was on top of your list leading with 60 points. That was 1.5
year ago in RGCC, remember?
As far as I am concerned criticism is meant and should be used to
improve
the quality of your work. It's then up to you to decide what to do with
the information.
A few months ago it became known that Rebel8 and Rebel9 were tested on
SSDF with its main book turned off. That was a huge mistake. I was not
very happy with this, still I saw no reason to have any doubts on the
creditability of your list. I understand these things sometimes happen.
But what is happening right now in unacceptable for me. I like to
compete on your list (still do) and have competed on your list since 12
years starting with Rebel 5.0 (5 Mhz). Here are my points....
#1. It's my opinion that every chess program that competes on your list
should be general available (see your own remark above)
#2. You made a first mistake on #1 (your own rule) when you accepted
to test Genius4 as a DOS version while the commercial program was
a Windows version. This DOS version of Genius4 was not available
for the public.
This information became publicly known after the Genius4 program
was on your list for a long time. I remember from RGCC you received
criticism for that.
You have given the program an unfair advantage above other
competitors on your list mainly because the DOS version could now
use much larger hash tables. We are talking about the days of 486
and P90 machines with 4 or 8 Mb memory.
#3. Now we have the Fritz5 case and it shows you haven't learned from
the Genius4 DOS mistake. You have ignored the criticism (and the
principals behind it) you received (mainly from RGCC) on the Genius4
DOS case where you sinned against your own rules.
In fact you have made the problem bigger. You accepted to
participate
a program that DEMANDS a 64 Mb machine so that it can use a big hash
table which gives the program an unfair advantage above other
programs
who have to play on 32 or 16 Mb machines. This is unfair
competition,
you may correct me if I am wrong.
Then I hear about the use of power books, endgame Cdroms which are
not part of the commercial package of the program. Minor points IMO
concerning the gain of some extra elo points. But another proof of
the tendency that you sin against your own rules.
#4. But my main criticism is that you have accepted an unknown piece of
software for autoplaying on your list. The autoplayer behaves
totally
different then the general accepted auto232 software. And then the
worst came, the new autoplayer is secret. Again you sin against
your own rules.
#5. You know very well that auto232 is a fragile piece of software and
easily can be miss-used still you accepted that other competitors
or other participants on your list were simply sent in the dessert
as nobody could check this piece of software.
#6. As a result of that (and this is not funny at all) I made a special
Rebel auto232 version which checks every auto232 opponent for fair
play. This version was sent to somebody who has the secret Fritz5
autoplayer. This special Rebel version reported, all ok.
Now for the future of your list I recommend you the following
improvements
which is a simple request to return to your basics and your own
rules....
#1. Only test programs as they come in the commercial package.
#2. Also test freeware programs like Comet, Crafty, Decade as they are
distributed. All these programs have in common that they are
generally available to the public. Nothing should be hidden.
#3. Do not accept any add-on's like separate books, endgame cdroms if
they don't come with the product.
#4. Do not accept autoplay software that is not public available. Stick
to the standard auto232 protocol all programmers have agreed on.
#5. Test programs on equal hardware. Do not accept special demands which
gives a program an advantage above others.
#6. Test with equal hash table sizes as far as this is possible. Do not
accept demands for a minimum hash table size other then the system
requirement that comes on the box of the program. Everything else is
a clear try to get an unfair advantage which you should not accept.
If you return to these rules again which were common in the past it will
be my pleasure to compete on your list again.
- Ed Schroder -
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.