Author: Vincent Diepeveen
Date: 04:23:40 05/05/04
Go up one level in this thread
On May 05, 2004 at 05:10:36, martin fierz wrote: >hi bob, > >rereading your DTS paper (you sent me a copy once), you reported 24 speedup >numbers for 4 processors (given in the end, for anybody interested). > >i get (using a black box): > >av. speedup: 3.65 >standard deviation of sample: 0.31 >standard error of average 0.064 > >so: average speedup(N=4) = 3.65 +- 0.07 would be a nice way to put this. i have all dts numbers in an excel sheet with search times and node counts. that's easier to do statistics with. >for those who don't have the paper, this was done on a cray, so it's not >comparable to crafty on an average N-way box you might have (and methinks this >experiment was done with cray blitz). > >this leads to two follow-up questions: >1) where does the 3.1 for crafty come from you usually quote? did you ever >publish a similar set of numbers for crafty? any .pdf / .ps to download for >that? where do the numbers 2.8 / 3.0 of vincent+GCP come from? how many >positions were in that test? 30 positions tested by GCP. I still have the outputs if you want them >2) can you give a similar error estimate for the 3.1 number (both std. dev and >std. error)? or even better, a full set of numbers so that i can do with them >whatever i want, since you seem so reluctant to compute std/ste? :-) >3) right, question 3 of 2 :-): you claimed somewhere deep down in the other >thread that it matters whether you look at related or unrelated positions. you >could prove/disprove this experimentally with a set of related positions (eg >from games of crafty on ICC) vs. a large test set (e.g. WAC). > >why is this important? without error estimates, you can discuss forever whether >2.8/3.0 are the same as 3.1. without hard data on 3) you can also discuss >forever whether the issue in 3) matters or not, and if it does, in what way and >how important it is. > >this is a simple experiment to do, and since my profession is about measuring >numbers i don't understand that you don't do it ;-) > >cheers > martin > > >results in table 4 for 4 processors: >3.4 >3.6 >3.7 >3.9 >3.6 >3.7 >3.6 >3.7 >3.6 >3.8 >3.7 >3.8 >3.8 >3.5 >3.7 >3.9 >2.6 >2.9 >3.8 >3.9 >4.0 >3.7 >3.8 >3.9
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.