Author: Jay Urbanski
Date: 16:40:06 06/27/03
Go up one level in this thread
On June 26, 2003 at 16:11:42, Robert Hyatt wrote: > >His testing methodology was not _that_ bad. He _did_ use the same compiler for >both processors, which is certainly reasonable. > >Whether he used that specific compiler because it made the g5 look better is >another issue, although it is doubtful that the gcc guys have got any great >g5 customizations built in yet. It's reasonable if all you are trying to measure is an Apples to Apples comparison of the hardware, and nothing else. If what you're tying to measure is what reasonable level of application performance a user can expect then I'd argue that it's not very reasonable becuase the quality of compilers available for the platform certainly needs to be taken into account. >One _could_ make a case for testing either way. (a) using the same compiler; >(b) using the _best_ compiler for each respective machine. > >The classic problem with (b) is that humans are influencing the outcome in a >big way, because you not only measure raw hardware performance, you measure how >good the optimizing gurus are at their craft. Either way is open to lots of >criticism, unfortunately. > >SPEC is still going to be the best comparison since each vendor is free to >use the fastest compiler and settings he can find, so long as the result >produces correct and validated answers. The "gurus" still count, of course, >but absolute is absolute. The official SPEC results go with (b) of course - which I would argue is how it should be. Yes humans influence the result but one has to assume that a vendor is going to expend some effort to optimize their platform to its fullest potential. If they're unwilling and/or unable to do that... then maybe that says something about the platform also. It may be the case that there is a lack of good optimized compilers for the PPC970 on OSX - which is why they used gcc. But there certainly is no lack of good compilers for Intel.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.