Author: Walter Faxon
Date: 00:54:18 01/05/06
Go up one level in this thread
On January 04, 2006 at 17:41:54, Charles Roberson wrote: > > I've never seen this theory stated before, if anyone in any of the science >communities has then I want the reference. If anyone has similar experience or >sees a flaw in my logic, lets hear it. > > Is it possible for an improvement in search depth to result in a performance >degradation in match play. > > I am thinking yes! The implication is interesting. You improve the search of >your engine. That is the only change. It now searches two ply deeper. But in >match play it scores worse. Your natural thought was that all else was the same >thus you've a bug in your search improvement. > > I think it is possible to improve the search and get worse results. Here is >how. > > Lets say that your position evaluator (PE) is out of tune on some >strategic/positional values. Deeper search works with the PE to create an edge >for your program. Your old search was keep pace (depth) with opponents, but the >new search sees two ply deeper on average. This gives your engine increased >opportunity to create an edge. Once the edge is realized, the engine is in a bad >position and the match is lost. > > Before it couldn't create the edge because it couldn't tactically out see >the opponents. Seems to me this scenario only happens when the PE is not >extremely out of tune, but is somewhat close to in tune. > > So, can increases in search depth in match play cause an out of tune PE to >reveal its issues. > > This seems to be happening in some my tests today. Other data my program >(prior) to the changes has a propensisty for getting into good opening and >middle game positions and then blowing it. Thus, increases in search depth may >allow it to see an advantageous postion (in its thoughts (PE)) and go for it at >earlier moves in the game. Also, increasing its chances of realizing those >positions. Thus, producing worse play. Dr. Hyatt has written here on a problem he had that relates to this issue (though I'm too lazy to look for the posts now; an account also appeared in the ICCAJ some time back). Maybe he can correct my memory of his story. It seems that his Cray Blitz program had a version written in fortran which his team used for testing since getting Cray time was difficult. The fortran version searched (IIRC) 5-7 ply. He noticed that his program tended to push pawns a lot, and since it didn't have much of a search this would create weaknesses, especially "pawn holes". So he added a term to the evaluator to effectively restrict these moves and the program clearly played better. Then an ACM CC championship came up and they got use of a Cray, which would increase their search depth to 8-9 ply. But the new version played badly and lost an early game to a weak opponent. Cray Blitz wasn't using its pawns! Bob removed the new evaluation term (I think he just added an instruction to jump around the eval code) and the old Cray Blitz made a comeback, defeating Hans Berliner's Hitech to win the Championship! But from seeing Cray Blitz's weak play in earlier games Berliner became suspicious and demanded an investigation. This was done later using another Cray to the satisfaction of then ICCA President Levy, if not Berliner. So yes, an evaluation term that looks good at one depth can bring you grief with deeper searching. Also IIRC, Hsu wrote that adding a term for "the initiative" hurt Deep Blue in testing. A good search already shows you what any initiative can do. And finally, if the success of Fruit is any example, "more" is not necessarily "better" in evaluation functions. It is best to only add rock-solid knowledge. At least regarding position evaluation. -Walter
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.