Author: Don Dailey
Date: 12:22:24 07/12/98
Go up one level in this thread
On July 12, 1998 at 13:00:23, Robert Hyatt wrote: >On July 12, 1998 at 11:43:09, Don Dailey wrote: > >>On July 12, 1998 at 05:45:36, Howard Exner wrote: >> >>>On July 11, 1998 at 16:31:41, Mark Young wrote: >>> >>>>On July 11, 1998 at 14:07:11, Howard Exner wrote: >>> >>>Other lines removed ... >>> >>>>>Some people tweak the bios settings on their motherboards (ram timings >>>>>as one example) which will make some machines different despite identical >>>>>processors. Typically a computer will have the original bios settings set >>>>>conservatively. >>>>> >>>>>I did find the original K6-233 timings on Ed's page were way off. I emailed him >>>>>what my machine found and he made the correction on his page. The original time >>>>>for the K6-233 was the time when the problem was solved (not the time Ed >>>>>wanted - the time when the ply was completed) so in this example it was >>>>>an error in following the directions for the test. >>>> >>>>That makes sense. I know you can tweak the bios settings but not by 20 to 30 >>>>percent in speed. I think this might be the cause of some other timing errors he >>>>has posted. If I run the test till rebel finds just the solution, the times >>>>matches up much better on the computers I have at home. >>> >>>The idea of waiting until the ply is complete may escape some >>>testers who normally just record the time to solution. >>>When Rebel 10 is released it might be a thought to revamp this computer >>>processor speed chart to include Rebel 10 and Decade 2.0, replacing Rebel 8 >>>and Decade 1.0. >>>I always enjoy these charts on how different processors compare on applications, >>>especially chess programs. Come to think of it, I've always been a glutton for >>>all kinds of Sports stats (I guess these computer charts are the same for me), >>>the funniest coming from the world of baseball... ie: so and so's batting >>>average on a full moon when Grandma's laundry is drying on the clothesline. >> >> >>Waiting for a ply to complete is how we do time tests on tactical >>positions too. We wait for the solution first, then for the iteration >>to complete. >> >>As far as I know, this was Larry Kaufman's idea. He noticed that >>the results of this method are much more consistant when comparing >>algorithm changes and one program against another. >> >>I don't view it as a major thing, just a slightly better way of >>doing things because it is more accurate. >> >>- Don > > >I think the "time to solution" is also a perfectly acceptable way of >tsting. In a game, I hardly ever "finish the last iteration" so such a >time doesn't mean anything. I do care about how long it takes me to find >a key solution, because if that time is within the time limit I would have >in a game, I would find it, if it isn't I won't. > >This is one reason why my parallel searches have *never* split work at the >root, (excepting the 2-week special edition we used in New York in 1983). >Splitting at the root will definitely have a longer time to solution when >there is not time to complete an iteration... > >So picking the time that the program finds the move (fail high) is a >reasonable way to time things, IMHO... IE this is the way everyone reports >WAC results, not waiting on the iteration to complete. If we did this, I >would not get wac141, because the fail high happens very quickly (a few >seconds) but getting the mate score back takes me about 2 minutes because I >get hung up in lots of deep checking lines... There is absolutely nothing wrong with time to solution and that is perfectly acceptable too. Larry likes time to solution iteration because it is less sensitive to root move ordering although its still not perfect. Our experiments seem to indicate that the best root move ordering for program strength is best to worst move, but the best ordering for solving problems is captures and checks first. When we made this improvement it hurt our solution times a little and Larry's method corrects this somewhat (but not all the way.) In our opinion it gives results more consistant with how good a program is and Larry always tests and compares various programs with this method for that very reason. But in the long run there is not that much difference in the two as long as you compare apples to apples. - Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.