Author: Vincent Diepeveen
Date: 10:29:11 09/06/02
Go up one level in this thread
On September 05, 2002 at 07:23:03, Georg v. Zimmermann wrote:
In the quoted case such as enron is
that when we talk about money and something goes wrong there are many
instances who keep themselves busy with such a problem.
Lawyers, prosecuters, financial guru's, government officials etcetera.
A major problem is that in science we usually talk about something where
there are only a hand full of specialists.
When we talk about parallel search there is perhaps 20 people world wide
who know the difference between using an smp_lock and not using it with
regard to a formula which gives a lineair speedup for crafty at n processors.
Usually there is no good control over scientists. Bob isn't the first
to invent numbers. Of course he did it very radical. He says the maximum
is 2.0 for his number (which is based upon a loaded hashtable, so
not a non deterministic problem already occurs even for single cpu
results) and he claims 2.0; rounded off or not it is a claim of 2.0 out of 2.0.
The sad thing is of course that from these 20 persons, not a single one
is good in statistics (me included). In fact some even challenge all
statistics. The few publications done on the parallel search terrain
are all real pathetic from statistical viewpoint.
person a showing a few 5 ply searches versus person b showing only 24
searches from a simplistic game (mchesspro-cray blitz).
Even though 24 is already much better than the previous researches it still
is statistically spoken very insignificant, because it is a game which
soon gets to a draw and then loss for white, so a win for black.
As some may realize it is hard to estimate possible positive or negative
effects of seeing a draw score onto speedups of a program.
There are other effects such as pondering and loading factor of the
hashtable, and not clearing it.
Many new effects in a research which aren't independable compared.
So not a single speedup number is comparable with other research.
Other researches again have the problem that they compare themselves
with dead slow programs, or programs which are completely different
(for example i remember a research of Feldmann who compared
a program A with very small hashtable versus a program B with big
hashtable, where all researchers know very well that a very small
hashtable is deadly for parallel speedup. In short a big hashtable
is a big advantage.
there is no clear standard in parallel research in short and
none of the published scientists, with exceptions perhaps of
Jonathan Schaeffer, seems to care much about it either.
I have heart when talking to different programmers who have a parallel
version, major criticism against results as published. I hope some
people who read this very well realize that i'm not only speaking for
myself here.
I get daily email of those programmers who themselves will never post
such things here, but completely agree with the data as published.
Very well known but hardly published is of course the biggest criticism
which is already posted more by me and others too, something bob clearly
did NOT do.
The Criticism has regard to that some programs in order to show a better
speedup were first completely raped and slowed down by magnitudes of factors,
before speedups were 'measured' or 'guessed' or 'extrapolated' even.
Obviously some things are hardware dependant. The alpha clusters where
zugzwang ran on (forgive me if it is other machines too) as well as
the sun hardware where Cilkchess ran on, it obviously is harder to communicate
there than at shared memory machines (and as we know there is difference
there too).
It doesn't take away that slowing down a program tens of times, that this
isn't a very good idea. Nevertheless the speedup looks very good.
Such things are simply not provable other than knowing what type of
program it is and knowing how fast similar programs can run on such
processors.
Let me give you for example the different cilkchess versions:
- single cpu cilkchess: 5000-10000 nodes a second
- single cpu other program (non cilk) from Don: 200000 nodes
a second. Of course it had no eval, but even then it'll get
100k nps.
Zugzwang was around 5000 nodes a second a cpu if i understand well.
Gnuchess, very well comparable in speed with zugzwang at the same
type of cpu's is (if not using 16 bits code but 64 bits code)
considerable faster. 100k nodes a second?
Of course this is all lineair slowdown.
However suppose i have at my pc a diep version getting 80k nps
a second at 1.6Ghz processor.
It's of course complete swindling IMHO to let it run at a 1.6Ghz
supercomputer processor which even is 64 bits at say 8k nps single
cpu there.
The best compare is the R14000 SGI chips. Single cpu i have been
busy *optimizing* the speed of diep for it the past 2 months. Though
not entirely finished yet, it gets single cpu at P2 about 25000
nodes a second without locking with 400MB hash.
That's with a very small hashtable.
Of course it will be a bit slower when using a big hashtable,
because that's memory from other nodes than this one.
Now suppose i use in all tests something which fits at the local
memory of 1 node (which is 2 GB).
So suppose i run it at 30 processors out of 32 and do not get
for the parallel version 25000 nodes a second, but like 10 to
20 times less so say 1250-2500 nodes a second and do all my
comparisions about the speedup i get with that 1250-2500
nodes a second.
Bob *never* did this.
Majority of the others did however.
I sure won't do it. I in fact don't even lose system time
to being multithreaded. I'm multiprocessor which is ideal at
such machines, because operations from node to node is more expensive
than at PCs.
In fact the old diep version already got about 500000 nodes a second
when i ran it at a few processors. However it took long to get that
speed. So for the latest version i have an additional CONDITION which
i am working. And that's that do not only want to get a great
speed out of it, i also want to get that speed within 90 seconds.
I do not know what these scientists feel they got paid for by their
universities, but i sure know i would not have paid them for their
results. If i would've been their boss, i would've fired them.
Nevertheless the mentality at universities is the opposite of this
mentality.
At computerchesstournaments the programmers there do not for nothing
refer sometimes to 'universitychessprograms'. If they do, it isn't
meant positively.
As i said cray blitz with respect to its speed definitely doesn't
belong in that category, but with respect to all facts posted i definitely
doubt it ever was good using the standards i refer to.
>Hello again,
>
>I agree that you have a strong point there.
>"Due process" is for the reasons you gave very important, and after reading
>your post I tend to give it more weigth in this matter than before.
>
>But IMHO one must still be very carefull that "due process" does not become an
>end in itself. If a shareholder tells the Public Prosecutor that he believes
>Enron CEOs "stole" his money do you want the Public Prosecutor put the
>shareholder in jail instead, because he did not use the right terminology,
>"theft" not applying to what they might have done ?
>
>
>I have the feeling it all boils down to how seriously you feel the mistake/facts
>jumbling/... Dr.Hyatt did was.
>In my oppinion it was rather severe, but I can accept other oppinions too.
>
>Then of course, there are the people who see the name "Vincent" and immediately
>start attacking him without even trying to understand what is going on, because
>maybe a long time ago he said a harsh word to them. For them, I can only feel
>pity.
>
>
>Georg
>
>
>
>On September 04, 2002 at 11:52:03, Roger D Davis wrote:
>
>>Sorry to scare your Georg. Whenever anyone is accussed of anything, there is
>>always something called "Due Process." Presumably, some methodology is at work
>>in Due Process to ensure that the facts are eventually uncovered. Bad taste
>>doesn't have a place in Due Process, in my opinion. We have Due Process because
>>because people sometimes jump the gun, because they make accusations without
>>gathering all the "facts," because the "facts" sometimes turn out to be wrong,
>>because they may have access to incomplete information which leads to the wrong
>>conclusion, despite their good intentions. And we also have Due Process because
>>people's reputations are damaged by such accusations, EVEN WHERE NO WRONG DOING
>>HAS OCCURRED.
>>
>>So I don't consider it ethical to co-opt this forum and fight it out, and ethics
>>is exactly what we are talking about here.
>>
>>You see, it is one thing to say that someone made a mistake, it is one thing to
>>say that they got their facts jumbled, it is one thing to say even that someone
>>is just an idiot. But charges of fraud are quite different in nature, because
>>they require knowledge about intentionality. Fraud goes quite far beyond whether
>>some report was wrong.
>>
>>Fraud is almost impossible to prove, so Vincent has set himself an incredibly
>>high hurdle to jump. I'm willing to bet that when the dust settles, it will
>>settle on Vincent.
>>
>>Roger
>>
>>
>>
>>Roger
>>
>>
>>
>>
>>
>>>Hi Roger,
>>>
>>>I am not taking side here but your post scares me a bit.
>>>
>>>Shouldnt we first look at content and then at presentation ? Vincents "bad
>>>taste" does not change facts at all.
>>>Nor do his motives.
>>>
>>>I can simply not understand nor follow the argumentation " Dr.Hyatt is a great
>>>guy and therefore we may not critisize errors/bad science/carelesness/fraud
>>>[select one] in his papers.
>>>
>>>
>>>Kind regards,
>>>
>>>Georg v. Zimmermann
>>>
>>>
>>>
>>>On September 04, 2002 at 01:34:18, Roger D Davis wrote:
>>>
>>>>Sorry Vincent, but this whole affair was presented in remarkable bad taste. In
>>>>my mind you've only damaged your own character and reputation: I find it odd
>>>>that someone who purports to have such extreme sensitivity to the truth could
>>>>have such insensitivity in regard to its presentation, and that certainly causes
>>>>me to doubt the sincerity of your motives.
>>>>
>>>>Roger
>>>>
>>>>
>>>>
>>>>On September 03, 2002 at 23:52:41, Vincent Diepeveen wrote:
>>>>
>>>>>On September 03, 2002 at 21:18:17, Roger D Davis wrote:
>>>>>
>>>>>sorry, but chess is an exact science. You have a speedup of 1.9 or you don't
>>>>>have a speedup of 1.9. It is very easy to measure. It is very easy to
>>>>>cheat by modifying it to 2.0 too.
>>>>>
>>>>>If it is modified, then sometimes statistical analysis can show that
>>>>>very clearly.
>>>>>
>>>>>In this case it does.
>>>>>
>>>>>Don't comapre with something that is not exact science please.
>>>>>
>>>>>You have a 2.0 speedup or you don't have a 2.0 speedup. hardware doesn't
>>>>>matter. Lies matter here. In this case modified search times to cover up
>>>>>a problem of 1-8 processors versus 16 processors.
>>>>>
>>>>>I am not here to tell you about the computer it ran on. I am here to
>>>>>show that there is a problem with the results written down.
>>>>>
>>>>>You can discuss results because: "such a good speedup can't happen",
>>>>>or "such a bad speedup is unexaplainable".
>>>>>
>>>>>But we can't discuss about this. It's a clear case of fraud. Nothing else.
>>>>>
>>>>>
>>>>>>I can tell you honestly that if I had to go back to my dissertation and
>>>>>>replicate my results, I doubt that I could do it. Just too much water under the
>>>>>>bridge. My memory is far to foggy to go back to all that data and all those
>>>>>>print outs, put everything back together again, and justify this or that
>>>>>>decision. My experience as a psychological researcher is that a lot of arbitrary
>>>>>>decisions are made on the way to some single statistic that presumably has
>>>>>>meaning. And then someone on your committee comes along and wants this or that
>>>>>>changed, and you do it because you need to show respect for senior professors,
>>>>>>although you might not agree at all. Every dissertation is the product of
>>>>>>compromise between a student and his committee. Likewise, most published
>>>>>>articles are the product of compromise between an author and the referees. The
>>>>>>process of science often introduces distortions that the author never intended,
>>>>>>including logical inconsistencies between one section of a document and another.
>>>>>>And that's just life.
>>>>>>
>>>>>>Roger
>>>>>>
>>>>>>
>>>>>>On September 03, 2002 at 20:48:08, Gian-Carlo Pascutto wrote:
>>>>>>
>>>>>>>On September 03, 2002 at 20:20:48, Roger D Davis wrote:
>>>>>>>
>>>>>>>>Wow, this comment is in exceptionally bad taste. You don't question the
>>>>>>>>scientific integrity of a researcher lightly, particularly in a public forum.
>>>>>>>
>>>>>>>I was responding to this post from Robert:
>>>>>>>
>>>>>>>---quote----
>>>>>>>
>>>>>>>[snip]... But that
>>>>>>>doesn't mean things were fabricated.
>>>>>>>
>>>>>>>But if you want to believe so, feel free. It doesn't change a thing either
>>>>>>>way...
>>>>>>>
>>>>>>>------------
>>>>>>>
>>>>>>>I don't know what exactly happend with the results. It seems from this thread
>>>>>>>that even Robert doesn't know. Just because of this, no matter how they were
>>>>>>>produced, I think they are questionable.
>>>>>>>
>>>>>>>--
>>>>>>>GCP
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.