Author: Richard A. Fowell (fowell@netcom.com)
Date: 20:16:02 02/08/98
Go up one level in this thread
On February 08, 1998 at 17:01:56, Robert Hyatt wrote: >On February 08, 1998 at 16:36:47, Richard A. Fowell (fowell@netcom.com) >wrote: > >>There is currently a round-robin tournament being conducted between >>Chessmaster 5500, Crafty 4.81, Fritz 5, Rebel 9 and HIARCS 6 >>on the Internet at a time control of 8 hours on a Pentium P5/133 >>(or equivalent). You can review the games in progress at the >>Gambitsoft site - http://www.gambitsoft.com/kup.htm. > >I hate to throw stones here, but this *really* needs checking into. IE >crafty is a clean 2.5X faster last time I compared a P5/133 to a P6/200. >Are the Crafty operators using the correct time here? I'd swear that I >have seen times that appear to be out of line with this, unless I >overlooked >something??? <snip> Certainly, the times used by the Crafty operators could have been chosen more carefully. A quick check (below) indicates that if the 2.5x factor is right, the current Crafty P6/200 times are perhaps 29% longer than they ought to be, which is perhaps a 18-22 Elo advantage. The handicaps I assigned were based on the data I had available, which was mainly my 33-problem HIARCS benchmark data. This data had the advantage of reduced variance due to using 33 problems, but obvious flaws because different programs benefit differently from different hardware. I mentioned that a time or two, I thought - in r.g.c.c. and in email. To cite one of my emails: > So far, I've been just using HIARCS speed to calibrate these machines. > Better would be to benchmark the program in question > (Crafty 14.?, CM5500, Fritz 5, where appropriate), but this would require > that this benchmark be run on the program in question on both a P5/133 > and the subject machine. Since I have neither a P5/133, nor any of the > non-HIARCS software, my plan is to benchmark using HIARCS as the yardstick > unless someone can test the other software on both the target machine > and the P5/133 for me. Since Fritz had "Fritzmark" data, I checked that, and Ed had some "Rebelmark" data I used. However, I didn't get much Crafty data. The only Crafty data I had handy was some 1. a3 data, which was mainly out of date, and suffered from being single-problem data to boot. Ideally, I'd get Crafty 4.8 data on the same 33-problem test (essentially, the Crafty team members and someone with a Pentium 133 could just run the LCTII test with 4.5 minutes/move and the KKKKup configuration with book off, and send me the log files. That test is about a 2.5 hour run. I'd look at the nodes searched for each of the 33 problem subset). I'm perfectly willing to do that (in the interests of science, if nothing else), but as far as changing the time handicaps at this point, that's a call for the TD. Wearing my other hat (as an operator of one of Crafty's opponents) I'm willing to go either way. If a P6/200 is 2.5x faster for Crafty than a P5/133, then the time for Crafty on a P6/200 should be: (8 hours)*(3600 sec/hour)/(2.5 factor) = 11520 sec. I have been giving them more time ... 14886 sec. If the 2.5x factor is right, the P6/200 Crafty's are getting 29% more time than they should, which is 0.37 log 2, or and 18-22 Elo advantage, if one believes in a 50-60 point per speed doubling in this situation (given the long time controls, one might expect less benefit for the added time.) The Crafty team is: THE CRAFTY team 1) Jason Deines jdeines@ibm.net P6 200 14,886 sec 2) Jan Frode Myklebust janfrode@ii.uib.no P5 200 20,367 sec 3) Jonathan Oei joei@mindspring.com P6 200 14,886 sec 4) Dann Corbitt dcorbit@solutionsiq.com P6 180 16,540 sec So, the only concern is for these machines vs. a P5/133 on Crafty 14.8.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.