Author: Robert Hyatt
Date: 12:02:13 09/06/02
Go up one level in this thread
On September 06, 2002 at 13:42:25, Vincent Diepeveen wrote: >On September 06, 2002 at 10:54:04, Robert Hyatt wrote: > >>On September 06, 2002 at 09:18:42, Vincent Diepeveen wrote: >> >>> >>>There is 2 speedups >>> a) number of nodes a second. We do not discuss this here >>> b) how much faster a program runs compared to a single processor. >>> >>>We do not discuss a) a lot. In fact a) is something bob doesn't print >>>at all at every ply. He prints it after a search. >>> >>>We discuss here about actual speedup b) : >>> >> >>+I+ discuss actual speedup. _you_ brought up the NPS question, not me. >>I don't consider NPS very interesting, other than as a "measure" of how >>fast things are going overall. > >obviously. single cpu it gets a million nodes a second here. dual >it'll never get far above 1.5 million nodes a second. 1.6 perhaps. Do I care about "here"? I get 1.9X the NPS on two cpus. Eugene got 1.95 on a dual itanium. Everybody else got above 1.8. Here is the results so far: dual ia64 733,jz 1.95X raw NPS dual PIII 450mhz 1.91X raw NPS quad xeon 700mhz 1.90X raw NPS (using only two processors) dual celeron 433mhz 1.86X raw NPS dual xeon 1700mhz 1.84X raw NPS dual AMD 1200mhz 1.68X raw NPS dual AMD 1730mhz 1.67X raw NPS The AMDs have problems. So what? You act like the worst case is the case we all have to use. I don't go that route. But even the AMDs show 1.67 or 1.68 times faster, and they are the _worst_ case machines so far... > >Of course all programs have more problems with dual AMDs here but >considering it is single cpu so much faster and cheaper than all >other smp processors it means in short it is interesting to tune >for it too. what does that have to do with anything we are discussing? Can we stick to NPS. or to raw speedup. Or to my speedup formula. Or _something_ without bouncing around like madmen? > >>>So 4 processors quad xeon 550 is compareable for bob with >>>a 2.8 times single cpu 550Mhz Xeon. > >>Or, according to the data I sent you via email, 4 processors quad xeon 700 >>is 3.0 times single cpu 700mhz xeon. But of course you are not going to ever >>use that number. And I call this "cherry picking". You have several >>observations. You _could_ average the two and say 2.9. But you didn't. You >>could give both, because you have both, but you didn't. You pick the worst one. >>And that is fraud itself... > >Most researchers use testsets to measure speedup. If you are the first >one to run an entire game, then the speedups which all other programmers >give nowadays are not comparable with yours. I run both. I have been running the positions _you_ wanted. You said "these are worse for speedup than test sets." Want to see the email? You said "on my machine you get a speedup of between 1.01 and 1.4..." Want to see the email? So I have been running that which you _asked_ me to run. Should I tell the ridiculous story about the "bad positions" that you didn't fix to see if I would catch them??? You have a test set you want me to run? Post it. I don't care. But I am sure that as soon as I get 1.6-1.8 on it with two processors, you will find yet another reason why that data is no good... > >The 2.8 speedup was from different positions, just like my own >measurements are done too at such different positions. > >We want a good compare to other programs. If you get a crafty speedindex of 100 >and i get a diep speed index of 90, without the 2 numbers being comparable >then it is a joke to write them down. I don't disagree with that. But _you_ do. Because in the past, _everybody_ has used the 24 kopec positions. That gives us a good comparison basis. But do _you_ want to be "standard"??? Nope. "Kopec is simple for speedup" and so forth, more hand-waving, and you run your own private positions. And you say you want to be able to compare with others??? Then why not either use a test set we all use, or post one and let everyone try it? rather than just tossing rocks here and there... > >Perhaps i should define the 'diep index' as being speedup number times 2. I had already assumed that your number was used as a seed to rand() anyway. > >So please compare the same things. You get 3.0 for 24 positions which >are all within the same game. Of course that's interesting in itself >and much better than 2 positions at a BK testset, but then i say interesting >because it is a bigger quantity of positions. Let's back up. Why did I run those 24 positions? How about an honest answer for a change? Did I run 'em because I wanted to make some kind of point? Or did you _ask_ me to run 'em? You keep saying "one position at BK", now "two positions at BK". I have published speedup results here more than once over _all_ (excepting position 1 a trivial mate) the BK positions. Bruce and I ran _several_ test sets as I was noticing the 1, 1.7, 2.4, 3.1 pattern... Not several positions, but several test sets... > >What we need is an objective compare between speedup numbers. If we can >agree upon that, then we can only make then the next step. All you have to do is pick one and _stick_ with it. You bounce all over the place, however. You got a test set I should run. Post and I'll give it a whack. > >Right now your numbers are speedups measured in a way no one else is >doing it! Are we talking Crafty or CB? If CB, the DTS paper is _very_ clear that the test was done in a different way. Because I wanted to measure a game. _you_ can do the same, but you should not use _my_ positions. You should play a game, then go back and re-run it with different numbers of CPUs just like I did. Then we _can_ compare game-time speedups even in different games... Or we can run the same test set and compare results on that test set. Or we can run multiple test sets and compare results there. It doesn't matter to me at all. The DTS article was done for a different reason... Not to compare to others, but to compare to myself, to see how much faster the 16 cpu version was in a real game... nothing more, nothing less... I was never really happy with the pondering approach, but it worked ok, even if it probably hurt my speedup a bit by giving the one cpu code more good hash info to work with. If you want to talk about flaws in the testing methodology, that's fine. I'm not going to talk further about flaws in the _data_ however. > >And with regard of cray blitz it's even worse, it's with a loaded hashtable >even. That's even more horrible. ANd when you think about it for any length of time, the loaded hash table helps the 1 processor test more than the 16 processor test, which is a bit of bias. I don't know how much but I could try it with Crafty and get a good estimate.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.