Author: Bruce Moreland
Date: 16:03:28 01/27/99
Go up one level in this thread
On January 27, 1999 at 13:33:42, Don Dailey wrote: >On January 26, 1999 at 17:52:25, Bruce Moreland wrote: > >> >>On January 26, 1999 at 16:25:17, KarinsDad wrote: >> >>>I'm glad that you are running other programs against the control. At what times >>>are you running the programs, on what type and speed processors, and what is >>>your matching criteria? >> >>One minute per move, you choose the processor, and a match is scored if you'd >>play the move at the end of the minute. >> >>I am flexible about the processor because I didn't want to split hairs over >>whether a P5/133 is X% slower than a P6/200 or whatever. I figured that a few >>people might run this on Crafty uing different hardware, and that might make >>show us what effect this had on match rate. >> >>This is a little too multivariate to make a good controlled experiment, but >>people will have reservations, possibly the same people, no matter what attempts >>are made to control the experiment better. I don't think it is possible to >>control it perfectly, so if you try to do so, people will point out the flaws >>anyway. >> >>bruce > >Yes, I agree with you completely as well as your last post. The only >think I would point out is that we need to run the Crafty version at >the same level and use the same "matching" criteria. What are we >considering a match? I proposed 2 ply and beyond at one point, Bob >says he took 3 minutes to 10 minutes and one of the testers posted >results that says he calls it a match only if his program wants to >play the move at the end of the 1 minute. What criteria did you >use? Time to find and hold until the end of a one-minute period. Same match criterion as for many of the other test suites. I figure people should have code for find-and-hold by now. And I would feel rather cheeky asking people for a week of CPU time on something like this. I'd rather have one minute on all platforms rather than one minute on a P6/200 and 25 seonds on a P2/450. And I wanted to do all of the games, rather than just one, because I wanted to see if some of the games had different match rates than others. >You last post hits the nail on the head, we compare match rates >against Crafty (under the same testing conditions) and compare. >But let's make sure we are using the same testing conditions. I don't think it matters how long people think, as long as it is repeatable. I was hoping to get results for a variety of hardware anyway. bruce > >- Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.