Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: This test is not scientific!

Author: Don Dailey

Date: 10:49:29 01/26/99

Go up one level in this thread


Hi Dann,

I'm not trying to prove guilt, I'm trying to prove innocence.  I don't
think this test can prove guilt but it can easily raise enough doubt
that we should back off.

If you are right and all or even one of the programs give a high match,
then there is plenty of room for doubt and we give them the benefit of
the doubt which I think is fair.   If however, Crafty matches
significantly higher percentage, then we have something to talk about.
I won't be claiming guilt in this case,  just that there is still
room to talk.

I have already discovered that running 60 seconds on Cilkchess will
guarantee a much lower match rate,  so I am a little concerned that
people will run this very short 60 second test and try to draw unfair
conclusions from it.

We will have enough data to come to some conclusion, even if the
conclusion is that we are not sure!   We also have log files from
them and an executable.  So we should be able to come to some
intelligent conclusion.   If nothing else, I want to know for my
own edification.


- Don




On January 26, 1999 at 13:18:58, Dann Corbit wrote:

>I don't think that any of these tests will prove anything.
>
>During the KKUP2, after producing a move and emailing it, I would start Crafty
>on the response to the move that Crafty suspected.  Only three or four times
>during the course of the game was an unexpected move delivered so that I had to
>redo the analysis.  That way, as soon as I got the move, I could deliver the
>response most of the time and speed up the game quite a bit.
>
>So my point is that excellent programs are going to play pretty much alike.
>
>On the other hand, I could fiddle with the eval function and make it play only
>slightly poorer, and have it disagree more often with only microscopic changes
>to the code (e.g. make a pawn worth 1.12 instead of 1 -- alter king safety a bit
>-- whatever).  Changing 1 (count it -- one) line of code could radically change
>play.  Changing 90% of the code might not.
>
>So the amount of recoding and the difference in playing style don't necessarily
>have any connection.
>
>So, in summary, it is going to be very difficult to show anything substantial
>with any amount of certainty.  And even if the programs matched 100% it would be
>possible that only a tiny fraction of the original code was used.
>
>The only way to know is to see the new code or to get and believe testimony from
>the one making the changes.
>
>At any rate, I am fairly certain that it would be illegal for bionic impakt to
>become a commercial program because of copyright violations.  Even the use to
>which it has been put is very questionable to me.
>
>On the other hand, I do not think that either party intentionally meant to harm.
> I think that the largest part of the problem is simply missing communication.
>Lots of unspoken/unwritten assumptions turned out to be incorrect (for both
>parties).
>
>I do hope that everything can be resolved to the satisfaction of all parties.



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.