Author: Melvin S. Schwartz
Date: 19:11:57 05/29/99
Go up one level in this thread
On May 29, 1999 at 15:59:32, Dave Gomboc wrote: >On May 29, 1999 at 14:05:29, Melvin S. Schwartz wrote: > >> >>On May 29, 1999 at 11:16:22, Dave Gomboc wrote: >> >>>On May 29, 1999 at 10:09:02, Melvin S. Schwartz wrote: >>> >>>> >>>>I don't understand how you can seriously give credence to this match when you >>>>are running Nimzo on superior hardware. The advantage of Nimzo on a Pentium 200 >>>>MMX is not to be taken lightly. Regardless, Hiarcs 6 is outdated by Hiarcs 7 and >>>>the Hash tables in Hiarcs 7 is much higher than what you listed for Hiarcs 6. My >>>>main point is that when testing chess programs, you should test them on the SAME >>>>type of computer. >>>> >>>>Regards, >>>>Mel >>> >>>No, he shouldn't. He should report the speed of the processor and the version >>>of the software, just as he has. >> >>If you support this kind of testing, good luck on trying to get meaningful >>evaluations. I think you're getting into more of a hypothetical circumstance >>here with uneven testing. > >Are you suggestions that more meaningful evaluations are achieved by a closed >group of 8 or so "newest version" programs on "latest" hardware? How will you >understand how good these are relative to older programs without >intergenerational competition? Besides, the larger player pool due to the >increased number of hardware/software combinations will provide more reliable >relative ratings for even the latest programs on the latest hardware. > >>>"Hiarcs 6, P90", "Hiarcs 7, P200MMX", and "Hiarcs 7, K2-450" are all different >>>entities that can be expected to have significantly different ratings. That a >>>newer hardware/software combination exists does not make it invalid or even >>>useless to assess the strength of an older one. >> >>I believe Nimzo 99 is a newer program than Hiarcs 6. If that is the case, it >>would futher support uneven testing. How many people would be interested in how >>Hiarcs 6 does against..as opposed to Hiarcs 7 against...?. Furthermore, who is >>still selling Hiarcs 6??? > >It doesn't matter if anyone is still selling it or not. Hiarcs 6/P90 still >provides an important performance benchmark for comparison. > >Even if this were not true, it is still the case that not everybody upgrades >their hardware and software every year. > >>I'm not saying there is absolutely no purpose in testing outdated software, but >>rather time and testing could be put to better use. > >I think the use it is being put to is very good. The person has a spare P90 >lying around, he might as well get some SSDF testing in. > >>Mel> > >Dave I will refer you to a posting by Robert Hyatt on 5/29 with the heading "Re: Uneven hardware". I will quote word for word, you can look at the posting if you don't believe what I quote: "If program A on hardware B beats program D on hardware E - does that say much about A compared to B? This belies the principles of science - you have to have a uniform platform for all participants to make any kind of judgement". Now, does that make it any clearer, or do you think Mr. Hyatt is also wrong? Hmmm?? By the way, my opinion posted on this matter was BEFORE I found Mr. Hyatt's statement quoted above. Mel
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.