Author: Tony Werten
Date: 08:47:39 09/04/02
Go up one level in this thread
On September 04, 2002 at 10:33:41, Robert Hyatt wrote: >On September 04, 2002 at 03:31:19, Tony Werten wrote: > >>On September 03, 2002 at 18:03:14, Gian-Carlo Pascutto wrote: >> >>>However reasonable your explanations may be, the gist of your DTS article >>>and the most important thing for comparison were the speedup numbers. After >>>what we discovered and what you just posted, it is clear that they are >>>based on very shaky foundations. >> >>I must be missing something. The only thing I see wrong with the speedup numbers >>is the way they are calculated. >> >>Since time was measured in miliseconds ie 3 significant numbers after the dot, >>the speedup should have been given in 3 significant numbers behind the dot as >>well. > > > >Just a note. I am not sure where this "millisecond" stuff comes from. But in >Cray Blitz, we only kept time to the nearest second, as produced by the Cray >library system call. So the speedup was calculated a different way and from that speedup the solutiontime was calculated. Unfortunately the solutiontime was in milliseconds,suggesting a high precision and the speedup factor only in tenths giving the strang looking "precision" of 2.000 or 1.900 for anyone calculating it the normal way from the tables. OK at least I understand what it's all about now. Not sure if I care though. Tony >For very short intervals, this wasn't very accurate, >but none of the searches here were short. This produced a problem only once, >when IM Larry Kaufman wanted to run some sort of rating test on Cray Blitz >at the 1993 ACM event. We solved almost every position in 0 seconds, and >that blew his rating formula out the roof. When I explained how the time >was kept, he decided to replace all the zeros by 0.5, because as I recall, >the time call we used did not round, it truncated to the second because of how >the cpu time was updated by the operating system... > >I am not going to say I didn't find a different system call at some point in >time, but I certainly don't remember doing so. And the few logs I have >remaining (all paper logs of some games played using a hardcopy printer) >certainly show this behavior... > >> >>Rounding it to tenths was simple wrong. Strange nobody noticed before, but not >>that important. The reason why nobody noticed is probably because these rounding >>rules are more part of the chemistry domain (or nature science). I guess >>mathematicians or computer science people are less aware of them. > >I think most of "us" (us being the ones writing parallel chess engines) >are aware that there is so much "fuzz" in the way a parallel search doesn't >produce repeatable results, that going beyond 1 decimel place becomes >pointless. > >For the Cray Blitz DTS article, I only had enough machine time to run each >test once. For my dissertation, I ran the 16 cpu tests dozens of times each >so that I could publish an "average" time and "average" speedup. > >To illustrate this, take Crafty and run a few positions using 1 cpu, then >run the same positions several times using 2 or 4 cpus. You'll see that the >number after the decimel will change significantly from one run to another. >Going to 3 digits of accuracy would simply produce some random digits to look >at... > > > > > >> >>Tony >> >>> >>>What's far worse, until you were directly accused, there was no indication >>>whatsoever for all the fiddling that was done with the auxiliary data. When >>>you were accused, you denied again, until other people supported Vincent's >>>point of view, when you suddenly got an email from an unknown person you're >>>not willing to disclose that 'refreshed your memory'. >>> >>>Additionally, the only other thing to support DTS, you PhD thesis, appears >>>to be basically totally unfindable for third parties. >>> >>>I hope you realize that a request from you to trust your numbers isn't >>>very convincing. In fact, with what we know now, I'm pretty sure the >>>article would never have gotten published in the first place. >>> >>>If Vincent wanted to discredit your results, then as far as I'm concerned, >>>he's succeeded 100%. >>> >>>-- >>>GCP
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.