Author: Robert Hyatt
Date: 18:06:30 05/04/04
Go up one level in this thread
On May 04, 2004 at 18:21:07, martin fierz wrote: >On May 04, 2004 at 13:44:21, Robert Hyatt wrote: > >>On May 04, 2004 at 10:49:42, martin fierz wrote: >> >>>On May 04, 2004 at 07:32:01, Rolf Tueschen wrote: >>> >>>>On May 04, 2004 at 07:11:15, martin fierz wrote: >>>> >>>>>On May 03, 2004 at 22:50:58, Robert Hyatt wrote: >>>>> >>>>>>If you recall, I _have_ given some error estimates in the past. Remember the >>>>>>wildly varying speedup numbers I showed you the first time this issue came up? >>>>> >>>>>i recall that you gave wildly varying speedup numbers, and an explanation for >>>>>why this happens. i don't recall a real error estimate, but that can be either >>>>>because >>>>>-> you gave one and i didn't see it >>>>>-> you gave one, i saw it and forgot >>>>>-> you didn't give one at all >>>>> >>>>>so... what kind of numbers would you give if you were pressed? >>>> >>>> >>>>Isn't it impolite to imply the third option if Bob JUST said that he did give >>>>some? >>> >>>no - asking questions always has to be allowed among scientists. forbidding to >>>ask questions is the hallmark of religious fanatics and fascists... but i >>>digress :-) >>>bob says he gave numbers, which he did. but IIRC, he never gave an error >>>estimate. so i am allowed to ask for it, and it is not at all impolite to do so. >>>what he did show is the speedup in about 30 different positions, which could >>>vary wildly depending on the position. >>> >>>i don't know why you think you have to stand up and defend bob every time >>>somebody says something about him you don't like. just leave that up to him. he >>>can take it :-) >>> >>>cheers >>> martin >> >> >>I wasn't offended. I hope my answer was ok. > >i didn't think you'd be offended, and your answer was ok, but...why don't you >take N (preferably N>>30...) positions and compute the standard deviation of >your speedup numbers, and the standard deviation of the average speedup? you can >still discuss the meaning of this, but at least you have an error margin you can >attach to your speedup. i don't see anything wrong with that!? even if the >probability distribution is obviously not a normal distribution, you can >probably approximate it as such, and get an idea of it's width from these >numbers. > >>This is not an easy question to deal with. > >>IE if you take the standard deviation of a set of random numbers between >>0 and N what do you get? That is what the speedup numbers look like for some >>positions. For others the speedup is a near-perfect constant value. Add some >>perfect constants plus some randomly distributed values and exactly what does >>the SD show? :) > >i don't quite understand your question. if you take enough positions, then you >will get something sensible, i would think. if you doubt this, you can take e.g. >10'000 sequential positions from crafty's ICC log, and bunch them together in >groups of 1000, and compute average speedup + stdev-of-average-speedup for each >of the bunches. i can't imagine that you get 10 wildly differing values, as your >statement above suggests. > >cheers > martin It isn't so easy to get speedup. IE how would I take a position that took X seconds with 2 or 4 cpus and compute the 1 cpu time? Think about it carefully and you will see the problem. How to get the 1-cpu test case to have a properly loaded hash table, killer move table, history table, etc, before starting the search??? The best bet is to take N positions where N is large. But then that is not the same as what happens in real games where the positions are connected via info passed from search to search in the hash table. It is just a very hard question to answer. And change the positions and you can change the answer significantly...
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.