Author: Robert Hyatt
Date: 17:14:49 09/03/02
Go up one level in this thread
On September 03, 2002 at 18:49:08, martin fierz wrote: >On September 03, 2002 at 18:26:09, Robert Hyatt wrote: > >>On September 03, 2002 at 17:44:26, martin fierz wrote: >> >>>On September 03, 2002 at 16:26:33, Robert Hyatt wrote: >>> >>>>On September 03, 2002 at 15:50:45, Matthew Hull wrote: >>>> >>>>>On September 03, 2002 at 15:42:08, Gian-Carlo Pascutto wrote: >>>>> >>>>>>http://sjeng.sourceforge.net/ftp/hyatt1.png >>>>>> >>>>>>I will try to upload the full article as soon as I can. >>>>>> >>>>>>-- >>>>>>GCP >>>>> >>>>>You've got to be kidding. When Vincent posts the numbers, he's got all these >>>>>trailing zeros. What's with that? >>>>> >>>>>It is Vincent that's faking numbers here, not Bob. Bob's numbers are just >>>>>rounded off. >>>>> >>>>>Vincent is the one emitting bogons here. >>>> >>>> >>>>If you didn't see my response elsewhere, this output was produced by a program >>>>I wrote to "eat" Cray Blitz log files. I did this for my dissertation as I >>>>produced tens of thousands of test position logs for that. >>>> >>>>I believe that it does something similar to what I do today, which means that >>>>anything between 1.71 and 1.8 is treated as 1.8, 1.81 to 1.9 is 1.9. >>> >>>that sure sounds like a bad thing to do! you should retain 2 digits, since >>>the interesting number is not the speedup (1.8, 1.9, whatever), but rather the >>>difference to 2. >> >>I don't use a "log eater" today. I generally run a set of positions, and >>get a time for one processor, then a time for 2 processors and divide the two >>by hand. I do round to one dec. place as the numbers are already very >>unstable, and going to more accuracy is really pointless... > >well, that is not the point! in the end, you give an average of the speedup >which turns out to be 2.0. now, even if your single numbers are unstable, the >average of them is much more stable, and could do with more than 1 digit. > >if you do your wrong rounding twice, first for the single measurement, and then >for the final result (you do, as i can see from the page GCP posted), then you >can actually make a larger mistake. take measurements 1.81, 1.81, 1.81 1.81 and >1.91. you round to 1.9 1.9 1.9 1.9 and 2.0. for the final result you round again >to 2.0. when the average of this thing was 1.83. of course, i have chosen my >numbers carefully, but there is simply no reason to do the rounding the way you >do. on average, with your double rounding, you are giving yourself a 0.1 speedup >which is not there! if you thought it was a good idea, and that it did not >affect the result really, you should have done it the other way round - down >instead of up. then at least you would be conservative... > >>> so if you measure 1.91 or 1.99, you really measure 0.09 or >>>0.01, and obviously these two numbers are quite different. >>>also, if you really feel like rounding, you should do it in the normal sense, >>>like 1.750 - 1.849 -> 1.8. i don't believe a word of vincent's post, but always >>>rounding upwards is definitely making your data look better than it is, and >>>should be rejected by any reviewer or thesis advisor - if he knew you were doing >>>that :-). not that it makes a big difference of course - but for the 2-processor >>>case we are talking about a potential >10% error, which where it starts getting >>>significant. >>> >>>aloha >>> martin >>> >> >>or really 5% max? ie 1.9 to 1.99 is only 5% difference... > >well, that's your interpretation: you say your are measuring x=1.9 or x=1.99. >but your experiment is designed to produce numbers between 1 and 2. so if you >look at it that way, your result is 1+x, delta_x being the .9 to .99 difference >which is 10%. this also makes sense from a practical point of view, since the x >thing is what you get from putting a second processor on the task. >even worse, i could argue that the interesting number is not 1.9 or 1.99 but >rather 2-x, with x now being how much inefficiency you get from the SMP search >overhead. and now we are talking about a factor of 10 in the result which can go >wrong. this is not a completely useless number either - obviously, an SMP search >which gets 1.95 on average is a MUCH better search than one which gets 1.8 on >average - even though it will not affect the end speed that much. but it is >obviously getting much closer to perfection than the 1.8 search. > >if you want to stay with your interpretation (which i think is a bad idea), then >you can still get nearly 10% error by taking 1.01 and 1.10 instead of your >numbers - not that that is very likely to happen :-) >whatever interpretation of your data you want to use, there is 1. no reason to >round the way you do, and 2. it is always better to keep a digit too much than >one too little :-) > >aloha > martin Here is the problem. I can run the _same_ position 4 times and get greater than 10% variance in the times. All using 4 cpus. All on a machine by themselves. Ugly, eh? And in that regard, .1 is not very significant, as I did mention in the paper...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.