Author: Uri Blass
Date: 15:09:29 09/12/03
Go up one level in this thread
On September 12, 2003 at 15:56:22, Bob Durrett wrote: >On September 12, 2003 at 13:39:55, Uri Blass wrote: > >>On September 12, 2003 at 12:56:56, Bob Durrett wrote: >> >>>On September 12, 2003 at 11:00:58, Matthew Hull wrote: >>> >>>>On September 12, 2003 at 08:43:02, Bob Durrett wrote: >>>> >>>>>On September 11, 2003 at 21:43:05, Robert Hyatt wrote: >>>>> >>>>>>On September 11, 2003 at 21:19:30, Bob Durrett wrote: >>>>>> >>>>>>>I have been comparing the top engines [on the same computer] by evaluating the >>>>>>>same position, sequentially, with each engine. Many examples so far, but in >>>>>>>every case the time required to get to a given search depth is MUCH longer for >>>>>>>Crafty. Why is that? Also, does it matter in terms of playing strength? >>>>>> >>>>>>Why is that? Different pruning rules. Commercial programs are doing some >>>>>>interesting forward pruning stuff. The only forward pruning I am using is >>>>>>null-move. >>>>> >>>>>I guess the "theory" is that successful pruning [forward or otherwise] is that >>>>>time is not wasted in evaluating inconsequential [i.e. irrelevant in the search >>>>>for the "best move"] moves and branches. >>>>> >>>>>The "64 Million Dollar Question" is: "How does a programmer know that his >>>>>pruning strategy is good?" The technical goal of such strategies must be to >>>>>decrease the time to reach the correct move while holding the probability of >>>>>error to within some acceptable limits. >>>>> >>>>>The answer must be in extensive testing. Maybe, too, some theoretical >>>>>considerations might help with finding the answer. Perhaps, ultimately, the >>>>>answer must be expressed statistically? >>>> >>>> >>>>> >>>>>Clearly, if a good pruning strategy exists, then the programmer is remiss if >>>>>he/she does not find and use it. >>>> >>>> >>>> >>>>Hey now, Bob D. Are you scolding the professor? I've always found it prudent >>>>not to underestimate (much less lecture) those with experience. >>>> >>>>MH >>>> >>>>P.S. "Old age and treachery will defeat youth and skill." >>> >>>I am older than Bob Hyatt, so I have more "experience" than he does. : ) >>> >>>On a more serious note, I am a USER of chess software as I have said here many >>>times. Currently, I am embroiled in doing a comparison of the top engines >>>[Shredder 7.0.4, Tiger 15, Deep Junior 8 (used on a single processor PC), Fritz >>>8, and Crafty 19.03.] This comparison is from a USER PERSPECTIVE. In each >>>case, I am using the engine as an analysis engine running under Chessbase 8 so >>>that differences in the native GUIs are transparent. >>> >>>The method of comparison is to analyze games [of a friend] where each move of >>>the friend's game is evaluated by each engine. I am allowing each engine to >>>reach a depth of no less than 16 for each move. I have analyzed about ten games >>>this way. [That's a lot of moves!] >>> >>>Certain patterns are becoming apparent: >>> >>>(1) It takes Crafty a lot longer to reach depth = 16 than the others. >>> >>>(2) Shredder consistently gives position evaluations which are INCONSISTENT >>>with those given by the other engines. [Shredder gives larger absolute values.] >> >>It means nothing. >>The meaning of +2 of shredder is simply different than the meaning of +2 of >>other engines. > >Uri, that IS useful information for me if you mean that the choice of the move >is not impacted. It confirms my intuition in the matter. > >There is another consideration. As a user, I wish to know whether a position >should be evaluated as -+, -/+, -/=, =, +/=, +/-, or +-. These symbols are in >widespread use in the published literature and their definitions have been >around long before chess engines and have nothing to do with the way chess >engines work. > >The programmer of each engine [or GUI?] must decide how to convert the numerical >position evaluation score to the appropriate evaluation symbol. Perhaps there >has been an unwritten convention adopted by most engine/GUI designers? For >whatever reason, Shredder 7.0.4 often says a position is +- when the other >engines are saying +/-. Similarly, Shredder says +/- when the other engines are >saying +/=, =, or even -/=. This is a real problem for a USER of chess analysis >engines because the user's subsequent decisions regarding the use of the >moves/lines can be thrown off the mark. I never care about the +/= or +/- I guess that only the author of the interface cared about it and he also did not consider it as important enough to do something more complicated than translating the evaluation to +/= or +/- I guess that the author of the engine also did not care about the meaning of the numbers and the only important thing is that bigger means better. It is possible that shredder means expected result of 70% for the stronger side when it says +2 when other programs mean 80% expected result for the stronger side. I do not care about it Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.