Author: Robert Hyatt
Date: 15:16:12 09/05/02
Go up one level in this thread
On September 05, 2002 at 13:28:20, Miguel A. Ballicora wrote: >On September 05, 2002 at 10:05:05, Robert Hyatt wrote: > >>On September 05, 2002 at 00:25:58, Miguel A. Ballicora wrote: >> >>>On September 04, 2002 at 18:38:17, Dann Corbit wrote: >>> >>>>My take on the matter (in one paragraph): >>>>Robert wrote a paper on parallel speedup, showing a 1.7 increase for 2 CPU's (as >>>>derived from his more general formula). Vincent was unable to reproduce this >>>>sort of speedup, and thought the research was faulty. Robert agreed that the >>>>test set was limited and you won't always get that sort of speedup, but as an >>>>average (over a broad set of positions) that's about what he got. There has >>>>been some acrimony over whether superlinear speedups are possible. I think that >>>>the jury is still out on that one. >>>> >>>>At any rate, that's my take on the whole thing. >>>> >>>>Vincent always sees things in pure, jet black or gleaming, powder white. If >>>>something isn't terrific, then it is pure junk. While I think his mode of >>>>interesting is a bit odd, it's one of the things that make Vincent interesting. >>> >>>He crossed the line when he used the word "fraud" and "lie" >>>to describe a scientific paper without any solid proof (he only proved a flaw in >>>the presentation). Too serious. >>> >>>To be honest, I am embarrassed to be reading this thread. One side does not >>>recognize a flaw (it could be honest and I believe it, happens many times, big >>>deal) and the other makes pathetic accusations of fraud mixing it up with old >>>issues (Deep blue etc.). To top it all, ad hominem attacks. >>> >>>In this conditions it is impossible to discuss anything. >> >>While I understand what you mean, I don't see any major "flaw". > >No, it is not major, it is a very minor flaw in the presentation. Not a big >deal, but you cannot stand there and say that it is just ok. You cannot say that >it is ok the way you rounded it and everything is justified by the big >variability. The only thing that the big variability shows is that the flaw is >minor, but it does not show that there is no flaw in the presentation. In those >cases standard deviations should be shown using numbers that were rounded >properly. Note that I believe that the numbers were rounded correctly now. The only output I have is from the initial "speedup" run of the log eater, which is a single sheet of paper. All the speedups are given as xx.x type numbers and since the log eater was written in FORTRAN, I am pretty sure that all the rounding was done by normal FP library rules... > >Don't get me wrong, I understand and accept completely everything you say, if I >were accused of fraud I would go overboard myself. But please, do not try to >convince us that those tables are the proper way to present something. >What I accept is that it does not change a bit the relevance of the ideas. The node table is the _only_ way the data can be presented. So I don't have much of an option to change that. The time should have been raw observed times. Why it is not, I can only speculate about. But the point is that the original observed times would definitely produce the speedups given, because they _did_. The re-constructed times _also_ produce the same speedups, so even though they are no longer raw/observed, nothing really changes.... But I certainly see your point... > >Regards, >Miguel > > >>As in the >>question I posed to vincent, if I ask you to measure the radius, diameter >>and circumference of a piece of pipe, once you have one, you can derive the >>others, but there will be a tiny accuracy issue. Because pi factors into >>the relationship among the three values, and what value do you use for pi? >>So even in something so simple as diameter vs circumference, there are a >>_set_ of possible diameters that will yield the same circumference, to some >>particular number of decimel places of accuracy. Yet in reality, only _one_ >>of those diameters can be the _real_ diameter. >> >>After showing Martin just how "bad" smp variability can be, that equates to >>a piece of pipe made out of some _very_ temp-sensitive material, so that the >>very act of touching it to measure it causes a change in the >>diameter/circumference. And that is what I see in SMP results _all_ the time. >> >>IE for a given actual time for a 16 processor test, and a computed speedup >>made by dividing the raw 1-cpu time by the raw 16-cpu time, you can now go back >>and re-compute the 1-cpu raw time. And there will be a _set_ of answers to >>that computation that still produce the same 16-cpu speedup. Is one of those >>pre-computed 1-cpu times (which can vary in the low order digits when the >>value is in the thousands) any better than another? Is one better because you >>computed it, and you actually observed it in the game under question? What if >>you run it again and you get _another_ one of the computed 1-cpu times, as it >>is easy to see the one-cpu time vary by a second or so over a long period, >>due to network traffic, interrupts, and so forth, maybe even a daemon process >>waking up for a moment to do something? >> >>My point? >> >>If you take a 16-cpu speedup, and a 16-cpu time, and use that to derive >>the set of 1-cpu times (measured only in seconds) you will get a set of >>N times that could produce that same speedup given the observed 16-cpu time >>and computed 16-cpu speedup. The inverse is the better way of thinking about >>this of course, because that 16 cpu time is just a "snapshot" that could vary >>dynamically, and as the 16-cpu time varies, so would the set of 1-cpu times >>that would still produce that same speedup. >> >>In other words, there is a _lot_ of inaccuracy already. So the question >>becomes, "why compute some of the times, why not observe them all?" That is >>"the" question here. And it is one I am not sure I can answer yet. I have >>gotten "closer" to the disk failure date. From info in my files at the office. >> >>I bought a Gateway G6/200 (pentium pro 200) when they first came out. I think >>Bruce got his a month or so before me. In early 1996, I had a disk crash, >>and called gateway for a replacement (this is the part I have docs for as you >>will see shortly). They sent one but when it arrived, it was a 68-pin wide >>scsi drive, while the machine they had sold me 2 months earlier was a 50-pin >>narrow scsi drive (this machine had 4 4.5 gig scsi drives). I called them to >>tell them about the "error" (I couldn't use that drive with my scsi controller) >>and they responded "we no longer have any narrow scsi stuff, we have migrated >>to wide scsi." I responded "great, so I have a machine 2 months old, that >>cost me $5,000, and you can't get me a replacement drive? Mark me off the >>gateway customer list (I used to order 20-30 machines _per year_ for our >>department labs)." That got their attention, they sent me three more wide >>scsi drives and a new controller. So I have at least homed in on when I lost >>"the world". And that suggests that we _had_ to reconstruct the data as best >>we could, although neither of us can say "yes that was the reason" with 100% >>reliability. I mainly remember trying to get Crafty back up to speed. I was >>getting ready for the WMCCC event, and was not releasing new versions. As a >>result, when the disk went down, I lost everything, and had to back up to the >>last released version and start over, trying to remember the changes I had >>made. >> >>I don't know when I started back to work on the dts paper, because the panic >>about the WMCCC and losing things is the thing that I _really_ recall (not >>to mention class assignments, handouts, etc.) But it is certainly within >>reason to think that led to the time computations. As I have said all along, >>the node computation was unavoidable from the start... >> >>So, as far as I am concerned, there is _nothing_ "false" in the DTS paper. I >>computed the speedups in the normal way, using raw data, back in early 1994 >>after the tests were run. That was the "critical data" presented in the paper. >>Later, we extrapolated the nodes when they were asked for, and it would seem >>that we extrapolated the times at the same time. >> >>That is about all I can remember, so far. I don't consider the "node" data to >>be very interesting at all, and the "time" data is dynamic enough due to the >>variability in speedup that any value I published, whether it be extrapolated or >>observerd, could likely _never_ be repeated again. As a new test run would >>certainly vary by seconds if not much more (another thread in this discussion >>shows some wild behavior on kopec 2, and some rock-solid behavior on kopec 3, >>to illustrate this point.) >> >>I don't see it as being a "fraud" at all, but then I'm not Vincent. I think >>the concept of "if you can't make yours look better, then make someone else's >>look worse" is extremely sad, of course. But he will continue to do that on >>all fronts. "crafty's eval is simple". "fritz knows shit about chess". >>"tiger is stupid in endgames." We've all seen those comments over and over. >>As well as "diep has the most sophisticated eval of any chess program" or >>"diep is the strongest program in the world at correspondence time controls". >> >>What more could anyone say? >> >>Will Crafty ever approach CB's DTS speedup? I doubt it, because I am not >>willing to give up the recursive search, which would be the first thing to >>go. If I had high expectations of running on a 32 cpu machine all the >>time, then I certainly might consider it. But I suspect that I will be >>using a quad for the foreseable future, with maybe an 8-way or 16-way >>box on rare tournament occasions. I'm old enough to try to avoid >>headaches, which you get with non-recursion when you try to read it later. >> >>Will someone else beat the DTS result? Most certainly. Even as I wrote the >>paper I noticed a couple of things that ought to be changed. Will it bother >>me when they do? Not at all. My EPVS algorithm beat the old PVS by a small >>amount. Schaeffer tried a new approach that was better than EPVS. Which led >>me to try something even better since I had a lot of time that I would be >>spending on my dissertation anyway, so that seemed like a natural project to >>undertake. Will I do that again? I don't think so. Too much development >>time. multiple years of debugging, including one full-time year as I finished >>the dissertation work and started the writing. The current SMP approach in >>Crafty is not half-bad. And it was not hard to write at all. Those two points >>say a lot... >> >> >> >> >> >>> >>>Regards, >>>Miguel >>> >>> >>> >>>> >>>>Robert has always been a man of strong convictions, and if you call him a >>>>'noo-noo head' he'll call you one back. He isn't one to back down when he >>>>thinks he is right. That's one of the things I like about Dr. Hyatt. >>>> >>>>When these two styles happen to ram into one another, the sparks are sure. A >>>>philosophical question is often asked: >>>>"What happens when an immovable object meets an irresistable force?" >>>> >>>>The 'debate' is an answer to that question. >>>>;-)
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.