Computer Chess Club Archives

Search

Terms

Messages

Subject: a number and two questions for bob

Author: martin fierz

Date: 02:10:36 05/05/04

hi bob,

rereading your DTS paper (you sent me a copy once), you reported 24 speedup
numbers for 4 processors (given in the end, for anybody interested).

i get (using a black box):

av. speedup: 3.65
standard deviation of sample: 0.31
standard error of average 0.064

so: average speedup(N=4) = 3.65 +- 0.07 would be a nice way to put this.

for those who don't have the paper, this was done on a cray, so it's not
comparable to crafty on an average N-way box you might have (and methinks this
experiment was done with cray blitz).

this leads to two follow-up questions:
1) where does the 3.1 for crafty come from you usually quote? did you ever
publish a similar set of numbers for crafty? any .pdf / .ps to download for
that? where do the numbers 2.8 / 3.0 of vincent+GCP come from? how many
positions were in that test?
2) can you give a similar error estimate for the 3.1 number (both std. dev and
std. error)? or even better, a full set of numbers so that i can do with them
whatever i want, since you seem so reluctant to compute std/ste? :-)
3) right, question 3 of 2 :-): you claimed somewhere deep down in the other
thread that it matters whether you look at related or unrelated positions. you
could prove/disprove this experimentally with a set of related positions (eg
from games of crafty on ICC) vs. a large test set (e.g. WAC).

why is this important? without error estimates, you can discuss forever whether
2.8/3.0 are the same as 3.1. without hard data on 3) you can also discuss
forever whether the issue in 3) matters or not, and if it does, in what way and
how important it is.

this is a simple experiment to do, and since my profession is about measuring
numbers i don't understand that you don't do it ;-)

cheers
  martin


results in table 4 for 4 processors:
3.4
3.6
3.7
3.9
3.6
3.7
3.6
3.7
3.6
3.8
3.7
3.8
3.8
3.5
3.7
3.9
2.6
2.9
3.8
3.9
4.0
3.7
3.8
3.9

This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.