Author: Gordon Rattray
Date: 11:17:30 07/26/00
I've been attempting to benchmark two different machines. So, I ran Junior 6a on a given position and saved the analysis. I repeated a check test *on the same machine* to see if the output would be any different, but I got the same analysis (as expected!). So I concluded that I had obtained some valid results for that particular machine. However, when I tried the same using Fritz 6a, it was difficult to get the same results twice! In fact, for one test position, during the first run it found the key move after about 2 minutes. When I ran it again, it required 6 minutes! Ok, so something is changing between test runs, but what? Here's some facts that may provide some clues... - the machine has a total of 64Mb RAM. 32Mb are being used for hash. There is no disk access during the test runs (I definitely watched for this!) - endgame table bases get accessed for some of my test positions - when I first start Fritz, I allow it an initial "dummy analysis run", just to get its hash table space setup. I discard this test run. - I'm using Windows 98 - before each test run, I perform a "clear hash tables" on the engine - during each test run, no other applications are being run on the PC, just Fritz. I'm also fairly certain that Windows doesn't start any background tasks such as it's scheduled system maintenance tasks. - when the test has run for long enough, I clip the analysis and only then run a word processor. I quit the word processor before performing the next run. I then run another "dummny analysis run" to try to allow Fritz to get it's memory space sorted out again (i.e. get any memory paging done), before starting the next test run. I don't expect results to be repeatable to the exact second every single time. Maybe some minor factors may throw things out slightly - fair enough. But I can't understand why I'm seeing such significant differences?? 2 minutes for one run, 6 minutes for the next! Has anyone else seen this? Or do you know what is responsible for changing the behaviour so drastically? I suspect my testing procedure doesn't support consistent runs, and despite trying to be careful, I'm still getting caught out?! Why is Junior not being affected similarly? (Finally, I apologise for not appending any "clipped analysis" here as I don't have the results on this machine. I also don't think that it will help identify the cause, but if this assuption is wrong, I'll forward the analysis a.s.a.p). Thanks in advance for any suggestions Gordon
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.