Author: Anthony Cozzie
Date: 09:31:47 05/07/04
Go up one level in this thread
On May 07, 2004 at 04:42:55, martin fierz wrote: >On May 07, 2004 at 03:17:44, Gian-Carlo Pascutto wrote: > >>On May 06, 2004 at 19:03:48, martin fierz wrote: >> >>>point #3 is perhaps most important for the bob vs vincent duel: the standard >>>error for a 4 CPU test run is on the order of 0.2. if vincent's tests were with >>>a similarly small number of positions, then the differences measured in these >>>experiments (2.8 / 3.0 / 3.1) are statistically insignificant, and the whole >>>argument is pointless :-) >> >>Not recessarily - disabling nullmove produced a result with the standard errors >>halved in my results. That would still allow a significant conclusion. >> >>If I assume your 0.2 is a 2SD number (95%), your results are compatible and >>running a non-nullmove test could still produce the same result. If 0.2 is a >>1SD number then for some reason your results were much more variable then mine. >> >> n speedup error (1SD) >>------------------------------------------ >>Nullmove 38 2.82 +- 0.101 >>No-nullmove 39 3.07 +- 0.056 >> >>-- >>GCP > >a repeat of "my" numbers > >4 CPUs: >3.15 +- 0.15 >3.29 +- 0.20 >3.06 +- 0.12 >3.19 +- 0.13 Something is amiss: Martin: 3.17 +- 0.17 GCP: 2.82 +- 0.10 Which means that from GCP's perspective martin's result is impossible (> 3 SD away!). Which leaves us with 3 explainations: 1. Crafty has changed 2. Crafty has a different speedup on Opteron vs P3 3. Speedup is not normally distributed. I am guessing 3. anthony
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.