Author: scott farrell
Date: 18:18:10 09/12/03
Go up one level in this thread
On September 12, 2003 at 12:56:56, Bob Durrett wrote: >On September 12, 2003 at 11:00:58, Matthew Hull wrote: > >>On September 12, 2003 at 08:43:02, Bob Durrett wrote: >> >>>On September 11, 2003 at 21:43:05, Robert Hyatt wrote: >>> >>>>On September 11, 2003 at 21:19:30, Bob Durrett wrote: >>>> >>>>>I have been comparing the top engines [on the same computer] by evaluating the >>>>>same position, sequentially, with each engine. Many examples so far, but in >>>>>every case the time required to get to a given search depth is MUCH longer for >>>>>Crafty. Why is that? Also, does it matter in terms of playing strength? >>>> >>>>Why is that? Different pruning rules. Commercial programs are doing some >>>>interesting forward pruning stuff. The only forward pruning I am using is >>>>null-move. >>> >>>I guess the "theory" is that successful pruning [forward or otherwise] is that >>>time is not wasted in evaluating inconsequential [i.e. irrelevant in the search >>>for the "best move"] moves and branches. >>> >>>The "64 Million Dollar Question" is: "How does a programmer know that his >>>pruning strategy is good?" The technical goal of such strategies must be to >>>decrease the time to reach the correct move while holding the probability of >>>error to within some acceptable limits. >>> >>>The answer must be in extensive testing. Maybe, too, some theoretical >>>considerations might help with finding the answer. Perhaps, ultimately, the >>>answer must be expressed statistically? >> >> >>> >>>Clearly, if a good pruning strategy exists, then the programmer is remiss if >>>he/she does not find and use it. >> >> >> >>Hey now, Bob D. Are you scolding the professor? I've always found it prudent >>not to underestimate (much less lecture) those with experience. >> >>MH >> >>P.S. "Old age and treachery will defeat youth and skill." > >I am older than Bob Hyatt, so I have more "experience" than he does. : ) > >On a more serious note, I am a USER of chess software as I have said here many >times. Currently, I am embroiled in doing a comparison of the top engines >[Shredder 7.0.4, Tiger 15, Deep Junior 8 (used on a single processor PC), Fritz >8, and Crafty 19.03.] This comparison is from a USER PERSPECTIVE. In each >case, I am using the engine as an analysis engine running under Chessbase 8 so >that differences in the native GUIs are transparent. > >The method of comparison is to analyze games [of a friend] where each move of >the friend's game is evaluated by each engine. I am allowing each engine to >reach a depth of no less than 16 for each move. I have analyzed about ten games >this way. [That's a lot of moves!] > >Certain patterns are becoming apparent: > >(1) It takes Crafty a lot longer to reach depth = 16 than the others. > As Prof Hyatt notes, less pruning, means more accurate on crafty's 16 ply search, so Crafty at 15 ply might be equivalent to some other at 16 ply. >(2) Shredder consistently gives position evaluations which are INCONSISTENT >with those given by the other engines. [Shredder gives larger absolute values.] Shredder is know to do lots more pruning/reductions. I am not compeltely sure, but they come from the same/similar camp to Rebel, and you can find some Rebel details here :http://members.home.nl/matador/chess840.htm Every time I try more futility/reductions I get better tactics, and weaker positional. As far as I know, Crafty strives for bug free/correct evals, and less for whipping other computer programs etc. > >(3) The "best move" selected by DJ is often different from that chosen by the >other engines. It often happens that the other engines all agree on the same >move. DJ is a "dark horse" in this respect. It should be noted that DJ jumps >to a higher depth very quickly, giving doubt as to what DJ's depth display >actually means [from the user perspective]. > >Bob D. > > >> >> >> >>> >>>Bob D. >>> >>> >>> >>>> (I assume you are not comparing junior in this mix since junior >>>>has a different meaning for "ply" than the rest of us). >>>> >>>> >>>> >>>>> >>>>>Could it be that Crafty's findings at depth = 12 are just as good as Junior's >>>>>findings at depth = 16? >>>> >>>>You simply can't compare depths. IE compare chessmaster to junior and then >>>>decide which is better based on that. :) >>>> >>>> >>>> >>>> >>>>> >>>>>If Crafty could find a way to do just as much useful computation in half the >>>>>time [on the same computer], it would play better. Right? [Obviously.] >>>>> >>>>>Maybe Crafty 19.03 is wasting too much time on non-essential >>>>>computations!!!!!!!!!!!! >>>>> >>>>>Bob D.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.