Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: None of these tests are truly scientific!

Author: Albrecht Heeffer

Date: 23:58:20 01/26/99

Go up one level in this thread


On January 26, 1999 at 16:29:15, Dann Corbit wrote:

>Darn, left out one more key question:
>
>How *repeatable* are these tests?  In other words, with 20 trials do you get the
>same answers each time?
>
>And yet another:
>What opening books are being used?  Is B.I.'s opening book identical to that
>used during the contest?  Is the version of B.I. used for these tests identical
>to that used in the contest (or are you simply using moves from that contest)?
>

The opening books on the download area of the Bionic website are the same
one we used in the Dutch Open. They have been slightly modified during
the tournament. When comparing moves played by Bionic and Crafty you
should exclude book moves.

>Do we know when B.I. went out of book?  It can probably be deduced from time
>spent, if we have exact timings for the moves (which I doubt is available).
>

You can see in the logfiles on the website exactly when Bionic went out
of book.

>I will again assert also, that similarity of play is a doubtful measure of the
>number of lines of code that are original.  Suppose that we had 100% agreement
>on each and every move.  Would that prove that the code was the same?  Suppose
>that we had only 30% agreement.  How would we know whether or not they simply
>changed the value of a knight by .3 and a pawn by .05 in eval?
>
>I think the only way you can really know is to see the code.

We did change a lot of values and parameters. The most important changes
however are in the prescan routines as explained on the website.

Albrecht Heeffer



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.