Author: Bob Durrett
Date: 09:56:56 09/12/03
Go up one level in this thread
On September 12, 2003 at 11:00:58, Matthew Hull wrote: >On September 12, 2003 at 08:43:02, Bob Durrett wrote: > >>On September 11, 2003 at 21:43:05, Robert Hyatt wrote: >> >>>On September 11, 2003 at 21:19:30, Bob Durrett wrote: >>> >>>>I have been comparing the top engines [on the same computer] by evaluating the >>>>same position, sequentially, with each engine. Many examples so far, but in >>>>every case the time required to get to a given search depth is MUCH longer for >>>>Crafty. Why is that? Also, does it matter in terms of playing strength? >>> >>>Why is that? Different pruning rules. Commercial programs are doing some >>>interesting forward pruning stuff. The only forward pruning I am using is >>>null-move. >> >>I guess the "theory" is that successful pruning [forward or otherwise] is that >>time is not wasted in evaluating inconsequential [i.e. irrelevant in the search >>for the "best move"] moves and branches. >> >>The "64 Million Dollar Question" is: "How does a programmer know that his >>pruning strategy is good?" The technical goal of such strategies must be to >>decrease the time to reach the correct move while holding the probability of >>error to within some acceptable limits. >> >>The answer must be in extensive testing. Maybe, too, some theoretical >>considerations might help with finding the answer. Perhaps, ultimately, the >>answer must be expressed statistically? > > >> >>Clearly, if a good pruning strategy exists, then the programmer is remiss if >>he/she does not find and use it. > > > >Hey now, Bob D. Are you scolding the professor? I've always found it prudent >not to underestimate (much less lecture) those with experience. > >MH > >P.S. "Old age and treachery will defeat youth and skill." I am older than Bob Hyatt, so I have more "experience" than he does. : ) On a more serious note, I am a USER of chess software as I have said here many times. Currently, I am embroiled in doing a comparison of the top engines [Shredder 7.0.4, Tiger 15, Deep Junior 8 (used on a single processor PC), Fritz 8, and Crafty 19.03.] This comparison is from a USER PERSPECTIVE. In each case, I am using the engine as an analysis engine running under Chessbase 8 so that differences in the native GUIs are transparent. The method of comparison is to analyze games [of a friend] where each move of the friend's game is evaluated by each engine. I am allowing each engine to reach a depth of no less than 16 for each move. I have analyzed about ten games this way. [That's a lot of moves!] Certain patterns are becoming apparent: (1) It takes Crafty a lot longer to reach depth = 16 than the others. (2) Shredder consistently gives position evaluations which are INCONSISTENT with those given by the other engines. [Shredder gives larger absolute values.] (3) The "best move" selected by DJ is often different from that chosen by the other engines. It often happens that the other engines all agree on the same move. DJ is a "dark horse" in this respect. It should be noted that DJ jumps to a higher depth very quickly, giving doubt as to what DJ's depth display actually means [from the user perspective]. Bob D. > > > >> >>Bob D. >> >> >> >>> (I assume you are not comparing junior in this mix since junior >>>has a different meaning for "ply" than the rest of us). >>> >>> >>> >>>> >>>>Could it be that Crafty's findings at depth = 12 are just as good as Junior's >>>>findings at depth = 16? >>> >>>You simply can't compare depths. IE compare chessmaster to junior and then >>>decide which is better based on that. :) >>> >>> >>> >>> >>>> >>>>If Crafty could find a way to do just as much useful computation in half the >>>>time [on the same computer], it would play better. Right? [Obviously.] >>>> >>>>Maybe Crafty 19.03 is wasting too much time on non-essential >>>>computations!!!!!!!!!!!! >>>> >>>>Bob D.
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.