Author: Miguel A. Ballicora
Date: 16:30:06 09/05/02
Go up one level in this thread
On September 05, 2002 at 18:16:12, Robert Hyatt wrote: >On September 05, 2002 at 13:28:20, Miguel A. Ballicora wrote: > >>On September 05, 2002 at 10:05:05, Robert Hyatt wrote: >> >>>On September 05, 2002 at 00:25:58, Miguel A. Ballicora wrote: >>> >>>>On September 04, 2002 at 18:38:17, Dann Corbit wrote: >>>> >>>>>My take on the matter (in one paragraph): >>>>>Robert wrote a paper on parallel speedup, showing a 1.7 increase for 2 CPU's (as >>>>>derived from his more general formula). Vincent was unable to reproduce this >>>>>sort of speedup, and thought the research was faulty. Robert agreed that the >>>>>test set was limited and you won't always get that sort of speedup, but as an >>>>>average (over a broad set of positions) that's about what he got. There has >>>>>been some acrimony over whether superlinear speedups are possible. I think that >>>>>the jury is still out on that one. >>>>> >>>>>At any rate, that's my take on the whole thing. >>>>> >>>>>Vincent always sees things in pure, jet black or gleaming, powder white. If >>>>>something isn't terrific, then it is pure junk. While I think his mode of >>>>>interesting is a bit odd, it's one of the things that make Vincent interesting. >>>> >>>>He crossed the line when he used the word "fraud" and "lie" >>>>to describe a scientific paper without any solid proof (he only proved a flaw in >>>>the presentation). Too serious. >>>> >>>>To be honest, I am embarrassed to be reading this thread. One side does not >>>>recognize a flaw (it could be honest and I believe it, happens many times, big >>>>deal) and the other makes pathetic accusations of fraud mixing it up with old >>>>issues (Deep blue etc.). To top it all, ad hominem attacks. >>>> >>>>In this conditions it is impossible to discuss anything. >>> >>>While I understand what you mean, I don't see any major "flaw". >> >>No, it is not major, it is a very minor flaw in the presentation. Not a big >>deal, but you cannot stand there and say that it is just ok. You cannot say that >>it is ok the way you rounded it and everything is justified by the big >>variability. The only thing that the big variability shows is that the flaw is >>minor, but it does not show that there is no flaw in the presentation. In those >>cases standard deviations should be shown using numbers that were rounded >>properly. > > >Note that I believe that the numbers were rounded correctly now. The only >output I have is from the initial "speedup" run of the log eater, which is >a single sheet of paper. All the speedups are given as xx.x type numbers >and since the log eater was written in FORTRAN, I am pretty sure that all >the rounding was done by normal FP library rules... > > > > >> >>Don't get me wrong, I understand and accept completely everything you say, if I >>were accused of fraud I would go overboard myself. But please, do not try to >>convince us that those tables are the proper way to present something. >>What I accept is that it does not change a bit the relevance of the ideas. > >The node table is the _only_ way the data can be presented. So I don't have >much of an option to change that. > >The time should have been raw observed times. Why it is not, I can only >speculate about. But the point is that the original observed times would >definitely produce the speedups given, because they _did_. The re-constructed >times _also_ produce the same speedups, so even though they are no longer >raw/observed, nothing really changes.... > >But I certainly see your point... You could (and should IMHO) have included a legend explaining how it was either calculated/extrapolated/measured etc. That's all. Regards, Miguel > > >> >>Regards, >>Miguel >> >> >>>As in the >>>question I posed to vincent, if I ask you to measure the radius, diameter >>>and circumference of a piece of pipe, once you have one, you can derive the >>>others, but there will be a tiny accuracy issue. Because pi factors into >>>the relationship among the three values, and what value do you use for pi? >>>So even in something so simple as diameter vs circumference, there are a >>>_set_ of possible diameters that will yield the same circumference, to some >>>particular number of decimel places of accuracy. Yet in reality, only _one_ >>>of those diameters can be the _real_ diameter. >>> >>>After showing Martin just how "bad" smp variability can be, that equates to >>>a piece of pipe made out of some _very_ temp-sensitive material, so that the >>>very act of touching it to measure it causes a change in the >>>diameter/circumference. And that is what I see in SMP results _all_ the time. >>> >>>IE for a given actual time for a 16 processor test, and a computed speedup >>>made by dividing the raw 1-cpu time by the raw 16-cpu time, you can now go back >>>and re-compute the 1-cpu raw time. And there will be a _set_ of answers to >>>that computation that still produce the same 16-cpu speedup. Is one of those >>>pre-computed 1-cpu times (which can vary in the low order digits when the >>>value is in the thousands) any better than another? Is one better because you >>>computed it, and you actually observed it in the game under question? What if >>>you run it again and you get _another_ one of the computed 1-cpu times, as it >>>is easy to see the one-cpu time vary by a second or so over a long period, >>>due to network traffic, interrupts, and so forth, maybe even a daemon process >>>waking up for a moment to do something? >>> >>>My point? >>> >>>If you take a 16-cpu speedup, and a 16-cpu time, and use that to derive >>>the set of 1-cpu times (measured only in seconds) you will get a set of >>>N times that could produce that same speedup given the observed 16-cpu time >>>and computed 16-cpu speedup. The inverse is the better way of thinking about >>>this of course, because that 16 cpu time is just a "snapshot" that could vary >>>dynamically, and as the 16-cpu time varies, so would the set of 1-cpu times >>>that would still produce that same speedup. >>> >>>In other words, there is a _lot_ of inaccuracy already. So the question >>>becomes, "why compute some of the times, why not observe them all?" That is >>>"the" question here. And it is one I am not sure I can answer yet. I have >>>gotten "closer" to the disk failure date. From info in my files at the office. >>> >>>I bought a Gateway G6/200 (pentium pro 200) when they first came out. I think >>>Bruce got his a month or so before me. In early 1996, I had a disk crash, >>>and called gateway for a replacement (this is the part I have docs for as you >>>will see shortly). They sent one but when it arrived, it was a 68-pin wide >>>scsi drive, while the machine they had sold me 2 months earlier was a 50-pin >>>narrow scsi drive (this machine had 4 4.5 gig scsi drives). I called them to >>>tell them about the "error" (I couldn't use that drive with my scsi controller) >>>and they responded "we no longer have any narrow scsi stuff, we have migrated >>>to wide scsi." I responded "great, so I have a machine 2 months old, that >>>cost me $5,000, and you can't get me a replacement drive? Mark me off the >>>gateway customer list (I used to order 20-30 machines _per year_ for our >>>department labs)." That got their attention, they sent me three more wide >>>scsi drives and a new controller. So I have at least homed in on when I lost >>>"the world". And that suggests that we _had_ to reconstruct the data as best >>>we could, although neither of us can say "yes that was the reason" with 100% >>>reliability. I mainly remember trying to get Crafty back up to speed. I was >>>getting ready for the WMCCC event, and was not releasing new versions. As a >>>result, when the disk went down, I lost everything, and had to back up to the >>>last released version and start over, trying to remember the changes I had >>>made. >>> >>>I don't know when I started back to work on the dts paper, because the panic >>>about the WMCCC and losing things is the thing that I _really_ recall (not >>>to mention class assignments, handouts, etc.) But it is certainly within >>>reason to think that led to the time computations. As I have said all along, >>>the node computation was unavoidable from the start... >>> >>>So, as far as I am concerned, there is _nothing_ "false" in the DTS paper. I >>>computed the speedups in the normal way, using raw data, back in early 1994 >>>after the tests were run. That was the "critical data" presented in the paper. >>>Later, we extrapolated the nodes when they were asked for, and it would seem >>>that we extrapolated the times at the same time. >>> >>>That is about all I can remember, so far. I don't consider the "node" data to >>>be very interesting at all, and the "time" data is dynamic enough due to the >>>variability in speedup that any value I published, whether it be extrapolated or >>>observerd, could likely _never_ be repeated again. As a new test run would >>>certainly vary by seconds if not much more (another thread in this discussion >>>shows some wild behavior on kopec 2, and some rock-solid behavior on kopec 3, >>>to illustrate this point.) >>> >>>I don't see it as being a "fraud" at all, but then I'm not Vincent. I think >>>the concept of "if you can't make yours look better, then make someone else's >>>look worse" is extremely sad, of course. But he will continue to do that on >>>all fronts. "crafty's eval is simple". "fritz knows shit about chess". >>>"tiger is stupid in endgames." We've all seen those comments over and over. >>>As well as "diep has the most sophisticated eval of any chess program" or >>>"diep is the strongest program in the world at correspondence time controls". >>> >>>What more could anyone say? >>> >>>Will Crafty ever approach CB's DTS speedup? I doubt it, because I am not >>>willing to give up the recursive search, which would be the first thing to >>>go. If I had high expectations of running on a 32 cpu machine all the >>>time, then I certainly might consider it. But I suspect that I will be >>>using a quad for the foreseable future, with maybe an 8-way or 16-way >>>box on rare tournament occasions. I'm old enough to try to avoid >>>headaches, which you get with non-recursion when you try to read it later. >>> >>>Will someone else beat the DTS result? Most certainly. Even as I wrote the >>>paper I noticed a couple of things that ought to be changed. Will it bother >>>me when they do? Not at all. My EPVS algorithm beat the old PVS by a small >>>amount. Schaeffer tried a new approach that was better than EPVS. Which led >>>me to try something even better since I had a lot of time that I would be >>>spending on my dissertation anyway, so that seemed like a natural project to >>>undertake. Will I do that again? I don't think so. Too much development >>>time. multiple years of debugging, including one full-time year as I finished >>>the dissertation work and started the writing. The current SMP approach in >>>Crafty is not half-bad. And it was not hard to write at all. Those two points >>>say a lot... >>> >>> >>> >>> >>> >>>> >>>>Regards, >>>>Miguel >>>> >>>> >>>> >>>>> >>>>>Robert has always been a man of strong convictions, and if you call him a >>>>>'noo-noo head' he'll call you one back. He isn't one to back down when he >>>>>thinks he is right. That's one of the things I like about Dr. Hyatt. >>>>> >>>>>When these two styles happen to ram into one another, the sparks are sure. A >>>>>philosophical question is often asked: >>>>>"What happens when an immovable object meets an irresistable force?" >>>>> >>>>>The 'debate' is an answer to that question. >>>>>;-)
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.