Author: Gian-Carlo Pascutto
Date: 04:06:21 11/22/02
Go up one level in this thread
On November 22, 2002 at 07:00:01, Omid David Tabibi wrote: >>Huh? You should never make such changes when you are already testing >>your algorithm! > >But if someone asks me to reproduce my results a year later? Save the version you used. I don't think this is even needed, see Roberts PhD :) >>Howso? The absolute times will be different yes. The ratios, not. >> > >But again, these are not the very same results. So what? Does that alter the conclusion of your paper? It better had not! >>> - Even on the same very hardware with the same very program, you will >>> not get the exact results of the previous test. >> >>Not exact, but close enough. Within 1% in my experience. >> > >"Not exact". Have you ever learned something about experimental error? This is one. Easily accounted for with _really_ basic statistics. >>Use a dedicated computer. If Windows is causing problems, use a Unix box. > >Haven't your university's UNIX servers run through any problem in the past six >months? So? Rerun the test in question. Besides, for testing I have a dedicated box that I run myself. Linux is free, a leftover computer isn't exactly hard to find either. >You see, fixed time poses many problems that have to bridged. But fixed depth >comparison, will result in the very same results, no matter what the hardware >is. That is how a scientific research should be conducted (as reproducible as >possible). You can do this perfectly with fixed time as well, your results will be perfectly within the statistical error. >Had I posted the time comparisons, you would have asked: >- what is your hardware? >- what is your program's NPS? >- is it a stable NPS? >- hasn't the computer crashed while conducting the tests? >- has it been a dedicated computer? >etc... You think so? Please take a look at the results I posted when I tested your algorithm. I never mentioned hardware, NPS, what computer or OS I used because they are irrelevant. >But now that I have presented my results in form of node counts, the above >questions are irrelevant. No, not at all. -- GCP
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.