Author: Ricardo Gibert
Date: 19:53:37 09/03/02
Go up one level in this thread
On September 03, 2002 at 20:00:03, Robert Hyatt wrote: >On September 03, 2002 at 19:02:55, Ricardo Gibert wrote: > >>On September 03, 2002 at 18:15:50, Robert Hyatt wrote: >> >>>On September 03, 2002 at 18:03:14, Gian-Carlo Pascutto wrote: >>> >>>>However reasonable your explanations may be, the gist of your DTS article >>>>and the most important thing for comparison were the speedup numbers. After >>>>what we discovered and what you just posted, it is clear that they are >>>>based on very shaky foundations. >>> >>>How so? The speedup numbers were _directly_ computed by dividing times. >>>Nodes were impossible to grab in the middle of a search so we computed what >>>they "should have been" and we did do some testing to be sure that the >>>estimation was very accurate... >> >>This is okay provided you *explicitly* and *clearly* state this is what you are >>doing and preferably with your justifications for doing this. Otherwise you're >>presenting of what they "should have been" is precisely the means by which >>fraudulent work is generated. If there was this omission, it definitely looks >>ugly even though it might be inadvertant. I hope you *explicitly* and *clearly* >>stated that you did this. Did you? > >I don't believe I did, no. Why? Because I am certain that if I _had_ been >able to grab actual node counts, they would have been virtually identical to >those I added. > >Also, the text was "done" and a referee wanted the results added after the >first cycle. I did so and added a mention, but it didn't seem appropriate >(at the time) to make changes elsewhere after other referees said "ok as is." Since it was added on to satisfy the request of the referee and it did not affect the conclusions substantively it is no big deal to me. It was not in your prior version and the obvious conclusion is the issue was of no great consequence to you. It should be viewed in that light, so I think people are making much ado about nothing here. Still, given the reactions here, it looks like you should have been more careful. 20/20 hindsight for sure though. > >However, as I also said, node counts are essentially meaningless in a parallel >search. Just crank up Crafty on a dual and give it a fixed depth to search a >position. Write down the nodes. Run it again. An example with 4 cpus: > > time=3.74 cpu=371% mat=0 n=4582031 fh=93% nps=1225k > time=3.60 cpu=372% mat=0 n=4446821 fh=93% nps=1235k > time=3.59 cpu=371% mat=0 n=4445337 fh=93% nps=1238k > time=3.60 cpu=373% mat=0 n=4473775 fh=93% nps=1242k > >Notice that there is a 1% variance in even a short search to 11 plies... >And even for time=3.6 seconds, nodes vary by 30K. The tiny error introduced >by calculating the node counts is lost inside that kind of error for really >counting them... :) or :( as the case may be. :) > >>> >>> >>>> >>>>What's far worse, until you were directly accused, there was no indication >>>>whatsoever for all the fiddling that was done with the auxiliary data. When >>>>you were accused, you denied again, until other people supported Vincent's >>>>point of view, when you suddenly got an email from an unknown person you're >>>>not willing to disclose that 'refreshed your memory'. >>> >>> >>>Sorry, but this was the first I heard about it. Vincent mentioned this about >>>a week ago, but gave _no_ specifics. And you are right, I didn't remember. I >>>am sure there are lots of other things I no longer remember either. But that >>>doesn't mean things were fabricated. >>> >>>But if you want to believe so, feel free. It doesn't change a thing either >>>way... >>> >>> >>>> >>>>Additionally, the only other thing to support DTS, you PhD thesis, appears >>>>to be basically totally unfindable for third parties. >>> >>>University microfilm will sell you a printed copy. It was published in >>>1988 at the University of Alabama at Birmingham. So far as I know, they >>>microfiche every dissertation published and provide copies for what cost >>>I don't know. I certainly bought more than one from them over the years, >>>but the last was so long ago I have no idea what they charged me for it. >>>(it was berliner's 1970 dissertation). >>> >>>> >>>>I hope you realize that a request from you to trust your numbers isn't >>>>very convincing. In fact, with what we know now, I'm pretty sure the >>>>article would never have gotten published in the first place. >>> >>> >>>I do not agree. The speedup numbers were verified by several different >>>people. The node counts were a issue that was not easily handled, but >>>someone wanted node counts shown because "everybody else had published >>>them previously, including my dissertation." >>> >>>I would certainly feel sure that the node counts are well within 99+% accurate. >>> >>>I'd be happy to run a short test on Crafty and do the same calculations to >>>show you why I am that certain... >>> >>> >>>> >>>>If Vincent wanted to discredit your results, then as far as I'm concerned, >>>>he's succeeded 100%. >>> >>>Fine. If that helps him produce a better speedup, good for him. But the >>>speedup number was absolutely produced from raw data... and the speedup was >>>what the article was all about. The other numbers were requested to be >>>compatible with other papers. >>> >>>That's all there was to it... >>> >>>Take 'em or leave 'em... >>> >>> >>> >>> >>>> >>>>-- >>>>GCP
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.