Author: Dave Gomboc
Date: 20:11:00 05/29/99
Go up one level in this thread
On May 29, 1999 at 22:11:57, Melvin S. Schwartz wrote: > >On May 29, 1999 at 15:59:32, Dave Gomboc wrote: > >>On May 29, 1999 at 14:05:29, Melvin S. Schwartz wrote: >> >>> >>>On May 29, 1999 at 11:16:22, Dave Gomboc wrote: >>> >>>>On May 29, 1999 at 10:09:02, Melvin S. Schwartz wrote: >>>> >>>>> >>>>>I don't understand how you can seriously give credence to this match when you >>>>>are running Nimzo on superior hardware. The advantage of Nimzo on a Pentium 200 >>>>>MMX is not to be taken lightly. Regardless, Hiarcs 6 is outdated by Hiarcs 7 and >>>>>the Hash tables in Hiarcs 7 is much higher than what you listed for Hiarcs 6. My >>>>>main point is that when testing chess programs, you should test them on the SAME >>>>>type of computer. >>>>> >>>>>Regards, >>>>>Mel >>>> >>>>No, he shouldn't. He should report the speed of the processor and the version >>>>of the software, just as he has. >>> >>>If you support this kind of testing, good luck on trying to get meaningful >>>evaluations. I think you're getting into more of a hypothetical circumstance >>>here with uneven testing. >> >>Are you suggestions that more meaningful evaluations are achieved by a closed >>group of 8 or so "newest version" programs on "latest" hardware? How will you >>understand how good these are relative to older programs without >>intergenerational competition? Besides, the larger player pool due to the >>increased number of hardware/software combinations will provide more reliable >>relative ratings for even the latest programs on the latest hardware. >> >>>>"Hiarcs 6, P90", "Hiarcs 7, P200MMX", and "Hiarcs 7, K2-450" are all different >>>>entities that can be expected to have significantly different ratings. That a >>>>newer hardware/software combination exists does not make it invalid or even >>>>useless to assess the strength of an older one. >>> >>>I believe Nimzo 99 is a newer program than Hiarcs 6. If that is the case, it >>>would futher support uneven testing. How many people would be interested in how >>>Hiarcs 6 does against..as opposed to Hiarcs 7 against...?. Furthermore, who is >>>still selling Hiarcs 6??? >> >>It doesn't matter if anyone is still selling it or not. Hiarcs 6/P90 still >>provides an important performance benchmark for comparison. >> >>Even if this were not true, it is still the case that not everybody upgrades >>their hardware and software every year. >> >>>I'm not saying there is absolutely no purpose in testing outdated software, but >>>rather time and testing could be put to better use. >> >>I think the use it is being put to is very good. The person has a spare P90 >>lying around, he might as well get some SSDF testing in. >> >>>Mel> >> >>Dave > >I will refer you to a posting by Robert Hyatt on 5/29 with the heading "Re: >Uneven hardware". I will quote word for word, you can look at the posting if you >don't believe what I quote: > >"If program A on hardware B beats program D on hardware E - does that say much >about A compared to B? This belies the principles of science - you have to have >a uniform platform for all participants to make any kind of judgement". >Now, does that make it any clearer, or do you think Mr. Hyatt is also wrong? >Hmmm?? By the way, my opinion posted on this matter was BEFORE I found Mr. >Hyatt's statement quoted above. > >Mel I am not claiming that "program A on hardware B beats program D on hardware E implies that program A is better than program D". You are trying to reach such a conclusion, and that is why you're at odds with everyone about this. Dave
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.