Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: test method lacks

Author: Robert Hyatt

Date: 11:47:39 09/04/02

Go up one level in this thread


On September 04, 2002 at 14:37:15, Vincent Diepeveen wrote:

>On September 04, 2002 at 14:18:38, Vincent Diepeveen wrote:
>
>another problem of your measuring speedups is that you cannot
>compare your own speedups on different positions even. There
>are 2 reasons:
>
>  a) the first position you start with a cleaned hashtables the
>     others you do not

So?  the first position out of book is necessarily going to have a clear
hash table.  Or not, if we are running in "tournament mode".  You do know
about "tournament mode" in crafty and Cray Blitz?  Where it computes the
set of known book moves, and then ponders from the rest so that if the
opponent plays an obvious (and not book move) we at least have a running
start on the computation.  You did know about that, right?  I have explained
it in the past.  The idea came from HiTech...  and Murray Campbell and has
been in Crafty forever...


>  b) positions where previous move you mispredicted the move
>     the speedup is not comparable with predicted moves. In fact
>     it's unclear even whether you add to it the time you searched
>     in permanent brain.


So what.  It is about the speedup in situations that happen _in_ _the_ _game_.
Some moves are predicted correctly.  They may (or may not) speed up better than
those that are not predicted correctly.  But since both happened in the _real_
game, the mixture certainly represents what happened in that game.

I explained my setup.  You could reproduce the test using your program if
you wanted.  Not that it would be particularly fair, since the positions
in the game were chosen by Cray Blitz and not your program, and your program
might not like them or handle them as well...




>  c) because of the opponent move 1 gets played after x time, move 2
>     after y time. In short it is not possible to make a scientific
>     compare from it
>



It is certainly possible to do so, because I did it.  If I did it,
anybody can do it.  Yes, it is more difficult to do.  But I don't consider
difficult to be a synonym for impossible.  You ought to try that sometimes.

By the way, I now notice all this flapping you are doing has finally convinced
a tropical storm to form in the Atlantic/gulf region.  I _knew_ that was
going to happen.



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.