Computer Chess Club Archives




Subject: Re: Verified Null-Move Pruning, ICGA 25(3)

Author: Gian-Carlo Pascutto

Date: 04:06:21 11/22/02

Go up one level in this thread

On November 22, 2002 at 07:00:01, Omid David Tabibi wrote:

>>Huh? You should never make such changes when you are already testing
>>your algorithm!
>But if someone asks me to reproduce my results a year later?

Save the version you used. I don't think this is even needed, see
Roberts PhD :)

>>Howso? The absolute times will be different yes. The ratios, not.
>But again, these are not the very same results.

So what? Does that alter the conclusion of your paper? It better
had not!

>>>    - Even on the same very hardware with the same very program, you will
>>>      not get the exact results of the previous test.
>>Not exact, but close enough. Within 1% in my experience.
>"Not exact".

Have you ever learned something about experimental error? This is one.
Easily accounted for with _really_ basic statistics.

>>Use a dedicated computer. If Windows is causing problems, use a Unix box.
>Haven't your university's UNIX servers run through any problem in the past six

So? Rerun the test in question. Besides, for testing I have a dedicated
box that I run myself. Linux is free, a leftover computer isn't exactly
hard to find either.

>You see, fixed time poses many problems that have to bridged. But fixed depth
>comparison, will result in the very same results, no matter what the hardware
>is. That is how a scientific research should be conducted (as reproducible as

You can do this perfectly with fixed time as well, your results will be
perfectly within the statistical error.

>Had I posted the time comparisons, you would have asked:
>- what is your hardware?
>- what is your program's NPS?
>- is it a stable NPS?
>- hasn't the computer crashed while conducting the tests?
>- has it been a dedicated computer?

You think so? Please take a look at the results I posted when
I tested your algorithm. I never mentioned hardware, NPS, what
computer or OS I used because they are irrelevant.

>But now that I have presented my results in form of node counts, the above
>questions are irrelevant.

No, not at all.


This page took 0.01 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.