Author: Christophe Theron
Date: 14:23:39 11/12/99
Go up one level in this thread
On November 11, 1999 at 09:44:23, Randy Schmidt wrote:
>I have to wonder how useful it is to compare Fritz6 on a Pentium 3-700
>vs. Chess Genius 2.0 on a Pentium 166!
>I suppose that one can calculate that the score difference should be
>12-3 and then compare the actual result, but isn't that like taking
>a high school track star and having him run the 100-meter vs. Carl Lewis
>but giving Carl Lewis a 15 meter head start?
>
>I would prefer to see results of various programs on equal hardware.
>Isn't that more to the point?
This question comes again and again. This should be put in a FAQ.
It is absolutely necessary for the SSDF (for example) to play these unequal
harware event matches.
The reason is very simple.
I can imagine that many people would like to see matches on K6-450 only, for
example. But if you do so, you'll get only an idea about the strength of the
programs relative to each other on this hardware.
But you'll have no idea about how they compare to the other hardware/software
combinations.
In other words, you'll be unable to give an elo rating to your K6-450 programs!
If you want to give accurate ratings to the programs running on a new, faster
hardware, it is necessary to play a lot of games against the "players" you have
already rated, that is against programs running on slower hardware.
So back to your question "how useful it is to compare Fritz6 on a Pentium 3-700
vs. Chess Genius 2.0 on a Pentium 166", the answer is that it is as necessary as
any other test. Because the point is not to see if A beats B, the point is to
measure by which percentage A beats B. And you need a lot of games to get a
decent approximation.
Christophe
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.