Author: Tony Werten
Date: 09:06:05 09/04/02
Go up one level in this thread
On September 04, 2002 at 11:47:39, Tony Werten wrote: >On September 04, 2002 at 10:33:41, Robert Hyatt wrote: > >>On September 04, 2002 at 03:31:19, Tony Werten wrote: >> >>>On September 03, 2002 at 18:03:14, Gian-Carlo Pascutto wrote: >>> >>>>However reasonable your explanations may be, the gist of your DTS article >>>>and the most important thing for comparison were the speedup numbers. After >>>>what we discovered and what you just posted, it is clear that they are >>>>based on very shaky foundations. >>> >>>I must be missing something. The only thing I see wrong with the speedup numbers >>>is the way they are calculated. >>> >>>Since time was measured in miliseconds ie 3 significant numbers after the dot, >>>the speedup should have been given in 3 significant numbers behind the dot as >>>well. >> >> >> >>Just a note. I am not sure where this "millisecond" stuff comes from. But in >>Cray Blitz, we only kept time to the nearest second, as produced by the Cray >>library system call. > >So the speedup was calculated a different way and from that speedup the >solutiontime was calculated. Unfortunately the solutiontime was in >milliseconds,suggesting a high precision and the speedup factor only in tenths >giving the strang looking "precision" of 2.000 or 1.900 for anyone calculating >it the normal way from the tables. > >OK at least I understand what it's all about now. Not sure if I care though. What I mean ( haven't read the paper yet ) is that I'm not sure if there is a difference between measuring the searchtimes and calculating the speedup and measuring the speedup and calculating the searchtimes. As long as the measuring is done correctly. Tony > >Tony > >>For very short intervals, this wasn't very accurate, >>but none of the searches here were short. This produced a problem only once, >>when IM Larry Kaufman wanted to run some sort of rating test on Cray Blitz >>at the 1993 ACM event. We solved almost every position in 0 seconds, and >>that blew his rating formula out the roof. When I explained how the time >>was kept, he decided to replace all the zeros by 0.5, because as I recall, >>the time call we used did not round, it truncated to the second because of how >>the cpu time was updated by the operating system... >> >>I am not going to say I didn't find a different system call at some point in >>time, but I certainly don't remember doing so. And the few logs I have >>remaining (all paper logs of some games played using a hardcopy printer) >>certainly show this behavior... >> >>> >>>Rounding it to tenths was simple wrong. Strange nobody noticed before, but not >>>that important. The reason why nobody noticed is probably because these rounding >>>rules are more part of the chemistry domain (or nature science). I guess >>>mathematicians or computer science people are less aware of them. >> >>I think most of "us" (us being the ones writing parallel chess engines) >>are aware that there is so much "fuzz" in the way a parallel search doesn't >>produce repeatable results, that going beyond 1 decimel place becomes >>pointless. >> >>For the Cray Blitz DTS article, I only had enough machine time to run each >>test once. For my dissertation, I ran the 16 cpu tests dozens of times each >>so that I could publish an "average" time and "average" speedup. >> >>To illustrate this, take Crafty and run a few positions using 1 cpu, then >>run the same positions several times using 2 or 4 cpus. You'll see that the >>number after the decimel will change significantly from one run to another. >>Going to 3 digits of accuracy would simply produce some random digits to look >>at... >> >> >> >> >> >>> >>>Tony >>> >>>> >>>>What's far worse, until you were directly accused, there was no indication >>>>whatsoever for all the fiddling that was done with the auxiliary data. When >>>>you were accused, you denied again, until other people supported Vincent's >>>>point of view, when you suddenly got an email from an unknown person you're >>>>not willing to disclose that 'refreshed your memory'. >>>> >>>>Additionally, the only other thing to support DTS, you PhD thesis, appears >>>>to be basically totally unfindable for third parties. >>>> >>>>I hope you realize that a request from you to trust your numbers isn't >>>>very convincing. In fact, with what we know now, I'm pretty sure the >>>>article would never have gotten published in the first place. >>>> >>>>If Vincent wanted to discredit your results, then as far as I'm concerned, >>>>he's succeeded 100%. >>>> >>>>-- >>>>GCP
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.