Author: Robert Hyatt
Date: 17:00:03 09/03/02
Go up one level in this thread
On September 03, 2002 at 19:02:55, Ricardo Gibert wrote:
>On September 03, 2002 at 18:15:50, Robert Hyatt wrote:
>
>>On September 03, 2002 at 18:03:14, Gian-Carlo Pascutto wrote:
>>
>>>However reasonable your explanations may be, the gist of your DTS article
>>>and the most important thing for comparison were the speedup numbers. After
>>>what we discovered and what you just posted, it is clear that they are
>>>based on very shaky foundations.
>>
>>How so? The speedup numbers were _directly_ computed by dividing times.
>>Nodes were impossible to grab in the middle of a search so we computed what
>>they "should have been" and we did do some testing to be sure that the
>>estimation was very accurate...
>
>This is okay provided you *explicitly* and *clearly* state this is what you are
>doing and preferably with your justifications for doing this. Otherwise you're
>presenting of what they "should have been" is precisely the means by which
>fraudulent work is generated. If there was this omission, it definitely looks
>ugly even though it might be inadvertant. I hope you *explicitly* and *clearly*
>stated that you did this. Did you?
I don't believe I did, no. Why? Because I am certain that if I _had_ been
able to grab actual node counts, they would have been virtually identical to
those I added.
Also, the text was "done" and a referee wanted the results added after the
first cycle. I did so and added a mention, but it didn't seem appropriate
(at the time) to make changes elsewhere after other referees said "ok as is."
However, as I also said, node counts are essentially meaningless in a parallel
search. Just crank up Crafty on a dual and give it a fixed depth to search a
position. Write down the nodes. Run it again. An example with 4 cpus:
time=3.74 cpu=371% mat=0 n=4582031 fh=93% nps=1225k
time=3.60 cpu=372% mat=0 n=4446821 fh=93% nps=1235k
time=3.59 cpu=371% mat=0 n=4445337 fh=93% nps=1238k
time=3.60 cpu=373% mat=0 n=4473775 fh=93% nps=1242k
Notice that there is a 1% variance in even a short search to 11 plies...
And even for time=3.6 seconds, nodes vary by 30K. The tiny error introduced
by calculating the node counts is lost inside that kind of error for really
counting them... :) or :( as the case may be. :)
>>
>>
>>>
>>>What's far worse, until you were directly accused, there was no indication
>>>whatsoever for all the fiddling that was done with the auxiliary data. When
>>>you were accused, you denied again, until other people supported Vincent's
>>>point of view, when you suddenly got an email from an unknown person you're
>>>not willing to disclose that 'refreshed your memory'.
>>
>>
>>Sorry, but this was the first I heard about it. Vincent mentioned this about
>>a week ago, but gave _no_ specifics. And you are right, I didn't remember. I
>>am sure there are lots of other things I no longer remember either. But that
>>doesn't mean things were fabricated.
>>
>>But if you want to believe so, feel free. It doesn't change a thing either
>>way...
>>
>>
>>>
>>>Additionally, the only other thing to support DTS, you PhD thesis, appears
>>>to be basically totally unfindable for third parties.
>>
>>University microfilm will sell you a printed copy. It was published in
>>1988 at the University of Alabama at Birmingham. So far as I know, they
>>microfiche every dissertation published and provide copies for what cost
>>I don't know. I certainly bought more than one from them over the years,
>>but the last was so long ago I have no idea what they charged me for it.
>>(it was berliner's 1970 dissertation).
>>
>>>
>>>I hope you realize that a request from you to trust your numbers isn't
>>>very convincing. In fact, with what we know now, I'm pretty sure the
>>>article would never have gotten published in the first place.
>>
>>
>>I do not agree. The speedup numbers were verified by several different
>>people. The node counts were a issue that was not easily handled, but
>>someone wanted node counts shown because "everybody else had published
>>them previously, including my dissertation."
>>
>>I would certainly feel sure that the node counts are well within 99+% accurate.
>>
>>I'd be happy to run a short test on Crafty and do the same calculations to
>>show you why I am that certain...
>>
>>
>>>
>>>If Vincent wanted to discredit your results, then as far as I'm concerned,
>>>he's succeeded 100%.
>>
>>Fine. If that helps him produce a better speedup, good for him. But the
>>speedup number was absolutely produced from raw data... and the speedup was
>>what the article was all about. The other numbers were requested to be
>>compatible with other papers.
>>
>>That's all there was to it...
>>
>>Take 'em or leave 'em...
>>
>>
>>
>>
>>>
>>>--
>>>GCP
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.