Author: Christophe Theron
Date: 09:44:27 04/29/98
Go up one level in this thread
On April 29, 1998 at 07:39:09, Kai Lübke wrote:
>On April 28, 1998 at 17:55:49, Len Spencer wrote:
>
>>
>>But Bert is right, how can you be absolutely sure both programs are
>>getting the same CPU time?
>
>For example by running two benchmarks on each program simultaneously.
>Example: to test CM 5500 vs. Fritz 5,
>have CM ponder over a position while you run the FritzMark test
>-> this will show you how much Fritz is slowed down by CM.
>Now do the same thing again and look at CM's node counter for, say, 30
>seconds
>-> this will show you how much CM is slowed down by Fritz.
>
>If the relation is not 50-50, you have to adjust the time controls
>accordingly to make up for that.
>Some programs are extreme CPU hogs even with PB off
>(CM 5500: slows others down to 70-80%;
> Nimzo98: slows others down to 40-50%).
>
>However, the above only works reliably with Windows programs, but not
>with all DOS programs.
>Example:
>
>Rebel is "frozen" in the back when you switch from DOS to Windows
>desktop, but if you switch back again, Rebel will be aware of the time
>that passed and will think it has pondered for these N minutes during
>which it was frozen.
>Consequently, if you freeze Rebel for 10 minutes, it will move instantly
>when you switch back to it, though it hasn't pondered a single node in
>the meantime!
>
>---
>Shep
You are making yourself your life very complicated, and your results
cannot be trusted. How can you be sure the benchmark function of a
program uses the same amount of CPU resource that the normal thinking
uses?
Just turn off permanent brain of both programs. Test DOS programs if
possible, preventing them from running in the background with the "DOS
box properties". They will get 99% CPU time when they think. That's all.
For Windows programs it doesn't work every time, depending of the
Windows program you are testing. If the Windows programs don't eat CPU
time when they are idle, it's OK. It seems to be OK with CM4000 for
example (except is some rare cases, for example when it opens a dialog
box, or when the message "side is checkmated" appears). It is not OK
with Genius5.
For those unpolite Windows programs, forget about any testing on one
computer only.
Christophe
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.