Author: Omid David Tabibi
Date: 04:18:48 11/22/02
Go up one level in this thread
On November 22, 2002 at 07:06:21, Gian-Carlo Pascutto wrote: >On November 22, 2002 at 07:00:01, Omid David Tabibi wrote: > >>>Huh? You should never make such changes when you are already testing >>>your algorithm! >> >>But if someone asks me to reproduce my results a year later? > >Save the version you used. I don't think this is even needed, see >Roberts PhD :) > >>>Howso? The absolute times will be different yes. The ratios, not. >>> >> >>But again, these are not the very same results. > >So what? Does that alter the conclusion of your paper? It better >had not! > >>>> - Even on the same very hardware with the same very program, you will >>>> not get the exact results of the previous test. >>> >>>Not exact, but close enough. Within 1% in my experience. >>> >> >>"Not exact". > >Have you ever learned something about experimental error? This is one. >Easily accounted for with _really_ basic statistics. > >>>Use a dedicated computer. If Windows is causing problems, use a Unix box. >> >>Haven't your university's UNIX servers run through any problem in the past six >>months? > >So? Rerun the test in question. Besides, for testing I have a dedicated >box that I run myself. Linux is free, a leftover computer isn't exactly >hard to find either. > >>You see, fixed time poses many problems that have to bridged. But fixed depth >>comparison, will result in the very same results, no matter what the hardware >>is. That is how a scientific research should be conducted (as reproducible as >>possible). > >You can do this perfectly with fixed time as well, your results will be >perfectly within the statistical error. > >>Had I posted the time comparisons, you would have asked: >>- what is your hardware? >>- what is your program's NPS? >>- is it a stable NPS? >>- hasn't the computer crashed while conducting the tests? >>- has it been a dedicated computer? >>etc... > >You think so? Please take a look at the results I posted when >I tested your algorithm. I never mentioned hardware, NPS, what >computer or OS I used because they are irrelevant. > Because we were discussing the performance on your specific program. But if we wanted to draw a wider conclusion regarding the algorithm, we had to know all those details. Besides, you might agree that the results you posted with fixed time, while being practical for testing your program, are not general enough to be publishable. >>But now that I have presented my results in form of node counts, the above >>questions are irrelevant. > >No, not at all. > >-- >GCP
This page took 0.42 seconds to execute
Last modified: Thu, 07 Jul 11 08:48:38 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.