Author: Uri Blass
Date: 10:39:55 09/12/03
Go up one level in this thread
On September 12, 2003 at 12:56:56, Bob Durrett wrote: >On September 12, 2003 at 11:00:58, Matthew Hull wrote: > >>On September 12, 2003 at 08:43:02, Bob Durrett wrote: >> >>>On September 11, 2003 at 21:43:05, Robert Hyatt wrote: >>> >>>>On September 11, 2003 at 21:19:30, Bob Durrett wrote: >>>> >>>>>I have been comparing the top engines [on the same computer] by evaluating the >>>>>same position, sequentially, with each engine. Many examples so far, but in >>>>>every case the time required to get to a given search depth is MUCH longer for >>>>>Crafty. Why is that? Also, does it matter in terms of playing strength? >>>> >>>>Why is that? Different pruning rules. Commercial programs are doing some >>>>interesting forward pruning stuff. The only forward pruning I am using is >>>>null-move. >>> >>>I guess the "theory" is that successful pruning [forward or otherwise] is that >>>time is not wasted in evaluating inconsequential [i.e. irrelevant in the search >>>for the "best move"] moves and branches. >>> >>>The "64 Million Dollar Question" is: "How does a programmer know that his >>>pruning strategy is good?" The technical goal of such strategies must be to >>>decrease the time to reach the correct move while holding the probability of >>>error to within some acceptable limits. >>> >>>The answer must be in extensive testing. Maybe, too, some theoretical >>>considerations might help with finding the answer. Perhaps, ultimately, the >>>answer must be expressed statistically? >> >> >>> >>>Clearly, if a good pruning strategy exists, then the programmer is remiss if >>>he/she does not find and use it. >> >> >> >>Hey now, Bob D. Are you scolding the professor? I've always found it prudent >>not to underestimate (much less lecture) those with experience. >> >>MH >> >>P.S. "Old age and treachery will defeat youth and skill." > >I am older than Bob Hyatt, so I have more "experience" than he does. : ) > >On a more serious note, I am a USER of chess software as I have said here many >times. Currently, I am embroiled in doing a comparison of the top engines >[Shredder 7.0.4, Tiger 15, Deep Junior 8 (used on a single processor PC), Fritz >8, and Crafty 19.03.] This comparison is from a USER PERSPECTIVE. In each >case, I am using the engine as an analysis engine running under Chessbase 8 so >that differences in the native GUIs are transparent. > >The method of comparison is to analyze games [of a friend] where each move of >the friend's game is evaluated by each engine. I am allowing each engine to >reach a depth of no less than 16 for each move. I have analyzed about ten games >this way. [That's a lot of moves!] > >Certain patterns are becoming apparent: > >(1) It takes Crafty a lot longer to reach depth = 16 than the others. > >(2) Shredder consistently gives position evaluations which are INCONSISTENT >with those given by the other engines. [Shredder gives larger absolute values.] It means nothing. The meaning of +2 of shredder is simply different than the meaning of +2 of other engines. It can be easily fixed by dividing the evaluation of shredder by a constant. > >(3) The "best move" selected by DJ is often different from that chosen by the >other engines. It often happens that the other engines all agree on the same >move. DJ is a "dark horse" in this respect. It should be noted that DJ jumps >to a higher depth very quickly, giving doubt as to what DJ's depth display >actually means [from the user perspective]. > >Bob D. Junior depth does not mean plies. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.