Author: Roger D Davis
Date: 09:49:01 12/18/01
Go up one level in this thread
> >Let's put it with some tact: > >1) if a program needs a special book to play correctly and cannot stand to use >the same book as its opponent, then it SUCKS. > >2) if a program is not able to perform well with ponder=off when its opponent is >also ponder=off, it SUCKS. > >I do not think 1 and 2 apply to Crafty. > >No matter what Bob tells, I have yet to see any proof that Crafty is handicapped >by ponder=off. As far as I remember, results have shown that Crafty does not >perform worse in ponder=on than in ponder=off matches. > >And this apply to most if not all chess engines. > >I also do not see any reason to believe that Crafty is more handicapped than >Fritz by a book that has not been designed specially for it. > >I could even say that a commercial program, which is supposed to be helped by a >hand tuned book, should be the most handicapped of the two. > > >I find Pavel's experiment interesting and I think it tells a lot about the >respective strength of Crafty and Fritz. I'm pretty sure additional experiments >will confirm this result, independantly of the time controls and book, and >ponder setting used. > >Those who reject the result do it for very strange reasons. Actually I think >they would reject the result of any experiment. In this world you need to be >able to draw conclusions (including margin of error in your conclusion) from an >unperfect experiment setup, using your own experience and understanding of the >experiment field. > > > > Christophe On the other hand, science itself is defined by its attempts to maintain what is known as "controlled conditions." Results of experiments are considered to be "ecologically valid" only to the extent that the conditions of the experiment resemble "real life." In real life, human players specialize in particular openings. They suck at other openings, either because these do fit the players temperament or style. SO they don't play them. Simply put, there is no moral imperative that a chess program play all books that can be put to it equally well. Chess programs will differ in their innerds, and it is these differences, presumably, that give the engine a particular style. If an author develops a book that plays the strengths of the engine while minimizing its weaknesses, then that make for smart tournament play. You may have different goals, Christophe, simply because your objective is to optimize the interaction of some set of chess principles across the universe of chess positions. But that's you...to say that a program that cannot stand to use the same book as its opponent sucks...in my opinion, that is extreme. In my opinion, your other arguments should also be evaluated carefully. You stated, "I also do not see any reason to believe that Crafty is more handicapped than Fritz by a book that has not been designed specially for it." Yet, it is also the case that if we were to systematically match various books with various engines and play a very, very large number of games, we would find that the win-loss ratio would change. So we can almost certainly conclude that some books will favor certain engines, while other books will handicap certain engines. Take any particular book and throw it at two engines in a match, and one may excel while the other suffers. Accordingly, you say, "I also do not see any reason to believe that Crafty is more handicapped than Fritz by a book that has not been designed specially for it." But I say, "I don't see any reason to believe that Crafty is EQUALLY HANDICAPPED OR AUGMENTED than Fritz by a particular book." The result is a loss of scientific control AND a loss of ecological validity, since the engines are intended to play with their books. And then there is the issue of pondering. Do all engines ponder with the same efficiency as they play? If engines vary in terms of their pondering efficiency, then of course pondering matters. Engines which benefit more from longer time controls are more likely to benefit from pondering. Accordingly, if you turn pondering off, then you are handicapping the engine. You say that you are pretty sure these results will be confirmed, yet you fail to say exactly what results it is to which you are referring. If you mean simply that Fritz is stronger than Crafty, then this will probably be confirmed. I do not believe that this is what you meant, however, since it is obviously too small a point for this forum. If you believe that the degree to which Fritz is stronger than Crafty will generalize across all time controls, all ponder settings, and all books, I find that suspicious, simply because I am asked to believe that there are no situations in which the interaction of book, ponder setting, and time control affect one engine more than another. I find that implausible. Roger
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.