Author: Rolf Tueschen
Date: 17:29:19 09/03/02
Go up one level in this thread
On September 03, 2002 at 17:30:35, Robert Hyatt wrote:
>As is usually the case, someone that helped me with this had sent an email
>while I was responding to the other posts. And when I read the second
>paragraph, it all "came back".
>
>Here is the issue:
>
>I started with a 16 cpu game log file. Note that this was from a real
>game. And in it I would find output just like Crafty's... Here is the
>idea:
>
> depth time eval PV
>
>followed by a summary.
>
>The problem is that the node count in the summary has nothing to do with the
>PV when it was displayed. The program _could_ have stopped the search as soon
>as the PV was displayed, or it could have stopped the search minutes later.
>As a result, I had no real node counts for the 16 cpu test that could be
>compared to anything else since there was no way to know when the 16 cpu
>test completed.
>
>We chose to do the following:
>
>1. run the positions thru a one processor search, and since there was no
>parallel searching going on, we could display an _exact_ node count for the
>one-processor test, as it would have been had the search stopped immediately
>after producing the critical PV move at the final depth. That value _is_ a
>raw data point.
>
>2. We then ran the positions thru the 2-processor search, taking the time
>for the same PV as the time. All the times are pure raw data, exactly. But
>we couldn't get a good node count. What we chose to do was to use an internal
>performance monitor we had built in, that very precisely told us how much cpu
>time had been spent playing chess by each processor. From these times, we
>computed speedups for 2 processors, 4, 8 and 16 (we didn't run the 16 cpu test
>again, we just used the raw log from the mchess pro game...
>
>3. We now had a set of speedups for each test. Which we plugged into the
>article. And again, it is important to note that for this data, the raw
>speedup was computed by dividing the times as you would expect.
>
>For the node counts, which was impossible for us to obtain from any but the
>one processor test, we simply extrapolated them based on the cpu utilization
>of all the processors. Some simple testing by searching to a fixed depth on
>one processor and then 16 processors shows that our "extrapolation" was "right
>on"... and we used those node counts.
>
>4. Clearly, the node counts are therefore produced from the raw 1-cpu data,
>multiplied by the percent of cpu utilization for the 2,4,8 and 16 cpu test
>cases. So they should correlate 100%.
>
>The only thing that my (nameless) partner said was that he could not remember
>if we did the same thing to produce the times since it would have been easier
>than trying to extract them from the logs later to produce the table for times.
>He "thought" that the times were added after a request from a referee, so that
>is possible.
>
>So, perhaps the data has some questionable aspects to it. The only part that
>I am _certain_ is "raw data" is the individual speedup values, because that is
>what we were looking at specifically. I had not remembered the node count
>problem until this email came in and then I remembered a case where Vincent
>was trying to prove something about crafty and got node counts suggesting that
>it should have gotten a > 2.0 speedup. I had pointed out that the way I do
>nodes, it is impossible to produce them anywhere except when all processors are
>idle, if you want an accurate number. I _should_ have remembered that we had
>the same problem back then. I am therefore afraid that the times might have
>been computed in the same way since it would have been quite natural to do
>so...
>
>I don't think this changes one iota about what is going on, of course. as
>given a speedup, and total time used by Crafty, I can certainly compute a
>node count that will be _very_ close to the real one. Which I supposed I should
>add so that Vincent can have his "every time the PV changes give me nodes"
>type of value.
>
>Keep in mind that this was an email from someone that worked on this with me
>back then. His memory was somewhat better because he actually wrote the code
>to solve the problem. But again, he was _very_ vague in remembering everything.
>It took a phone call for us to discuss this to get as far as I did above. I
>might remember more as time goes on.
>
>But the bottom line is "trust the speedup numbers explicitly". And if you
>trust them, the others can be directly derived from them. For 16 cpus, Cray
>Blitz generally searched 100% of the time on each cpu. If it produced a speedup
>of 16, then each cpu searched 1/16th the total nodes searched by one processor.
>If it produced a speedup of 8, then each cpu searched 1/8 of the nodes searched
>by one processor, which is 2x the total nodes, aka search overhead.
>
>Sorry for the confusion. Stuff done 10 years ago is difficult enough.
>Remembering the "log eater" was harder since I didn't write all of it...
>
>Bob
Bob,
would you be so kind to explain from your standpoint (and also in the light of
the adding here) how Vincent's conclusions could be understood? I mean, you made
a highly reasonable impression with your answers except the ugly 'medication'
chapter. (Please note that I do not support Vincent's vocabulary either when he
spoke of lies and mass fraud, as if it would have been proven yet; to Vincent:
you might find strange things but the proof of intentional fraud and or lying is
certainly something else!)
Would you admit, that from what Vncent could read in your article ('THe Page'),
he might have forcedly come to his (wrong) conclusions?
Why am I writing this?
Bob, if you want it or not, for the young here you are a 'fixed star', also
because no one else is showing up from the past. I remember how it became clear,
and you wrote it, that Vincent could impossibly reach a sound performance with
the super computer in the actual event. So, his frustration seems explainable.
Also, please note, that Vincent isn't a scientist yet. He's a good chessplayer,
but by far not IM or GM. His good chess surely is a big motivation for him, but
I fear that his 'science' is not based on a sound education yet. As far as I am
informed. He might be an excellent programmer, but this still doesn't help to be
a sound scientist. I can say that because Vincent made a lot of unneccessary
verbal attacks, which are completely unknown in real science.
But why I make this appeal to you? Simply because I at your age and with your
status and your successes, I would be happy if I had a young man writing and
searching the way Vincent is doing it. I wouldn't feel insulted at all. And I
saw that you also stayed cool most of the time. Sure, it'salso not really nice
that Vincent gets on your throut after he cooperated with you to his own
benefit. But then I remember how the also young T. Kerrigan communicated with
you. Also in a very impolite manner. And you did _never_ complain. So,please
let's give Vincent the same chance. I for one would be happy if my old articles,
if I were you or some collegue of the past, would be checked so carefully no
matter with what motivation. Even you had the chance now to remember some
long-time forgotten stuff.
I think that even if people make grave mistakes, we should stay cool and explain
the basics, in special if we are teachers!
So, please you should reconsider the first verdict about Vincent. He's still
worth your helping hand. The details might be better discussed in private email
between you two.
To Vincent:
Your original article is one of the best here. Also if your conclusions are not
soun d and the language is too impolite. But I can almost feel the temperament
and the fever that is burning in you.
Please try to get a few lessons in science, I mean the basics, logic and stuff
like that. The chain is very feable: your undoubtable observations, the
impossibility to reproduce something, your reading of the article, your
interpretation then, very limited as many here have told you, and then the
conclusions with intentional fraud and conscient lies. This is not what is
happening very often in science! But errors, sloppiness, confusion, zeitnot,
influences from outside (helping hands) often happen, that's true. But the proof
that Bob commited all those "crimes" is not there! The main mistake in your
article is the short time you had to make such objections and that you surely
didn't show the article to good scientists before! Next time you should do that.
I mean scientists not students. People with experience. Because if you don't do
that, then you might fall into trivial traps. Here, I guess but don't know for
sure, the question of the data-eater. Only one point. Of course there could be
mistakes in Bob's article, but even that wouldn't necessarily mean that he
cheated in his dissertation. BTW did you check this one too?
And still I think that your article is very important. It gives us a good
description of the mess you are in since you began your work on the super
computer for the Wch. If you can overcome the actual difficulties you are a good
candidate for future titles! Please try to preserve your "fever", but please
reduce the heavy artillery words in your vocabulary. In science it's enough if
you present your data and make hypotheses. The rest, possible ugly conclusions
is a natural consequence then. No need to push your case like attorneys in
American courts.
GENS UNA SUMUS
Rolf Tueschen
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.