Author: Robert Hyatt
Date: 14:19:21 07/12/98
Go up one level in this thread
On July 12, 1998 at 15:22:24, Don Dailey wrote: >On July 12, 1998 at 13:00:23, Robert Hyatt wrote: > >>On July 12, 1998 at 11:43:09, Don Dailey wrote: >> >>>On July 12, 1998 at 05:45:36, Howard Exner wrote: >>> >>>>On July 11, 1998 at 16:31:41, Mark Young wrote: >>>> >>>>>On July 11, 1998 at 14:07:11, Howard Exner wrote: >>>> >>>>Other lines removed ... >>>> >>>>>>Some people tweak the bios settings on their motherboards (ram timings >>>>>>as one example) which will make some machines different despite identical >>>>>>processors. Typically a computer will have the original bios settings set >>>>>>conservatively. >>>>>> >>>>>>I did find the original K6-233 timings on Ed's page were way off. I emailed him >>>>>>what my machine found and he made the correction on his page. The original time >>>>>>for the K6-233 was the time when the problem was solved (not the time Ed >>>>>>wanted - the time when the ply was completed) so in this example it was >>>>>>an error in following the directions for the test. >>>>> >>>>>That makes sense. I know you can tweak the bios settings but not by 20 to 30 >>>>>percent in speed. I think this might be the cause of some other timing errors he >>>>>has posted. If I run the test till rebel finds just the solution, the times >>>>>matches up much better on the computers I have at home. >>>> >>>>The idea of waiting until the ply is complete may escape some >>>>testers who normally just record the time to solution. >>>>When Rebel 10 is released it might be a thought to revamp this computer >>>>processor speed chart to include Rebel 10 and Decade 2.0, replacing Rebel 8 >>>>and Decade 1.0. >>>>I always enjoy these charts on how different processors compare on applications, >>>>especially chess programs. Come to think of it, I've always been a glutton for >>>>all kinds of Sports stats (I guess these computer charts are the same for me), >>>>the funniest coming from the world of baseball... ie: so and so's batting >>>>average on a full moon when Grandma's laundry is drying on the clothesline. >>> >>> >>>Waiting for a ply to complete is how we do time tests on tactical >>>positions too. We wait for the solution first, then for the iteration >>>to complete. >>> >>>As far as I know, this was Larry Kaufman's idea. He noticed that >>>the results of this method are much more consistant when comparing >>>algorithm changes and one program against another. >>> >>>I don't view it as a major thing, just a slightly better way of >>>doing things because it is more accurate. >>> >>>- Don >> >> >>I think the "time to solution" is also a perfectly acceptable way of >>tsting. In a game, I hardly ever "finish the last iteration" so such a >>time doesn't mean anything. I do care about how long it takes me to find >>a key solution, because if that time is within the time limit I would have >>in a game, I would find it, if it isn't I won't. >> >>This is one reason why my parallel searches have *never* split work at the >>root, (excepting the 2-week special edition we used in New York in 1983). >>Splitting at the root will definitely have a longer time to solution when >>there is not time to complete an iteration... >> >>So picking the time that the program finds the move (fail high) is a >>reasonable way to time things, IMHO... IE this is the way everyone reports >>WAC results, not waiting on the iteration to complete. If we did this, I >>would not get wac141, because the fail high happens very quickly (a few >>seconds) but getting the mate score back takes me about 2 minutes because I >>get hung up in lots of deep checking lines... > >There is absolutely nothing wrong with time to solution and that is >perfectly acceptable too. > >Larry likes time to solution iteration because it is less sensitive >to root move ordering although its still not perfect. > >Our experiments seem to indicate that the best root move ordering >for program strength is best to worst move, but the best ordering >for solving problems is captures and checks first. When we made this >improvement it hurt our solution times a little and Larry's method >corrects this somewhat (but not all the way.) In our opinion it >gives results more consistant with how good a program is and Larry >always tests and compares various programs with this method >for that very reason. > >But in the long run there is not that much difference in the two >as long as you compare apples to apples. > >- Don ' I'm not sure this solves the root-move-ordering problem however, because if you look at checks first, and a check is best, the rest of the moves will still go by a whole lot faster once you get alpha set. With null- move, it is even worse, as it is not uncommon for the first move to take 90% of the total search time, and the null-move search waxes the rest of the moves in a few seconds... So you still would get buggered by funny move ordering I would think. however, I think that in general, this has nothing to do with how well a program plays chess anyway, only with how well it can solve a tactical problem. But against good players (GM players) I rarely find wild tactical wins, rather Crafty has to simply play chess, convert a small advantage into a bigger one, and eventually win, perhaps without using a single check extension on the critical moves... Of course it has to avoid tactics as well, but I don't think it is necessary to have a huge tactical search, so long as you don't go nuts... You do have to have a good tactical search or you will get bombed by folks like Yasser, Anand, Shirov and others... But beating them in a tactical crush is not common...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.