Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Proposal: New testing methods for SSDF (1)

Author: Robert Hyatt

Date: 09:42:23 04/14/98

Go up one level in this thread


On April 14, 1998 at 07:49:54, Ed Schröder wrote:

>On April 14, 1998 at 07:12:34, Amir Ban wrote:
>
>>On April 14, 1998 at 03:50:49, Ed Schröder wrote:
>>
>>>Everybody who wins the GOLD medal has to pass the doping control.
>>>And the current gold medal winner *REFUSES* to pass the doping
>>>control.
>>>
>>>Openness is the only way that will take away all suspicion. In the
>>>meantime the BASIC trust from my side is gone because the autoplayer
>>>is kept hidden by all means. Here it comes again what bothers me the
>>>most....
>>>
>>
>>To use your analogy, you would give every athlete the right to stop
>>every other athlete and demand a drug test ?
>>
>>You know these tests are done by the Olympic committee or other official
>>bodies. What if every athlete would say: "Hey ! I and my doctor are
>>experts in doping ! We know of 60 drugs the Olympic committee hasn't
>>heard of. Anyway I don't trust the committee. I want the gold medalist
>>tomorrow morning in my doctor's office."
>>
>>It's up to the organization doing the tests (SSDF) to make sure that
>>everything is done in good order. You know they can get any advice they
>>want if they ask for it (including from you). If the autoplayer is an
>>important user feature, let the buyers disapprove (to your advantage).
>>In your position, you are the wrong person to complain about this.
>>
>>If you won't trust the SSDF on this issue, why would you trust them on
>>any other, including the list itself ?
>>
>>Amir
>
>
>This has nothing to do with MIS-TRUST of the SSDF. They have my FULL
>trust. I told you my complaint, only programmers can (are able) to
>check if an autoplayer is clean. You did not comment on that.
>
>Concerning the used gold medal example, it is normal in every sport
>to pass the doping control. Generally excepted. No demands should
>be needed.
>
>The use of AUTO232 by SSDF exist by the permission of the
>programmers. If this rule is broken then what do you expect?
>
>Silence?
>
>- Ed -


Amir makes a good point... if folks demand auto232, they'll get it.  If
they don't they won't.

The SSDF has other problems to deal with as well...  IE when a testing
methodology becomes very well known, everyone can take some steps to
take
advantage when they see a small "crack" in the procedure.  IE the claim
that a program would "hang" if it came out of book with a really
negative
eval, which would time out and restart without counting that result.

Whether that happened or not is not nearly so important as the fact that
it *could* happen.  As could other things as you play around with
auto232
and find cute things to try...  what would happen if you sent a stream
of
null characters over the serial port when you are not on move?  Would
the
interrupts be significant enough to noticably slow the other program?
On
a P5/90?  There are probably plenty of things that can be exploited to a
lesser or greater extent, *when* the testing/experimental setup is known
by everyone...

Simplest case is to play A vs B and tune A to beat B.  But it could get
much worse.  All because the SSDF list has become quite important to the
consumer, which means it is also important to the sellers.  This is
right
out of "marketing 101" I'm afraid.  :)




This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.