Author: Uri Blass
Date: 01:00:45 01/05/06
Go up one level in this thread
On January 05, 2006 at 03:54:18, Walter Faxon wrote: >On January 04, 2006 at 17:41:54, Charles Roberson wrote: > >> >> I've never seen this theory stated before, if anyone in any of the science >>communities has then I want the reference. If anyone has similar experience or >>sees a flaw in my logic, lets hear it. >> >> Is it possible for an improvement in search depth to result in a performance >>degradation in match play. >> >> I am thinking yes! The implication is interesting. You improve the search of >>your engine. That is the only change. It now searches two ply deeper. But in >>match play it scores worse. Your natural thought was that all else was the same >>thus you've a bug in your search improvement. >> >> I think it is possible to improve the search and get worse results. Here is >>how. >> >> Lets say that your position evaluator (PE) is out of tune on some >>strategic/positional values. Deeper search works with the PE to create an edge >>for your program. Your old search was keep pace (depth) with opponents, but the >>new search sees two ply deeper on average. This gives your engine increased >>opportunity to create an edge. Once the edge is realized, the engine is in a bad >>position and the match is lost. >> >> Before it couldn't create the edge because it couldn't tactically out see >>the opponents. Seems to me this scenario only happens when the PE is not >>extremely out of tune, but is somewhat close to in tune. >> >> So, can increases in search depth in match play cause an out of tune PE to >>reveal its issues. >> >> This seems to be happening in some my tests today. Other data my program >>(prior) to the changes has a propensisty for getting into good opening and >>middle game positions and then blowing it. Thus, increases in search depth may >>allow it to see an advantageous postion (in its thoughts (PE)) and go for it at >>earlier moves in the game. Also, increasing its chances of realizing those >>positions. Thus, producing worse play. > > >Dr. Hyatt has written here on a problem he had that relates to this issue >(though I'm too lazy to look for the posts now; an account also appeared in the >ICCAJ some time back). Maybe he can correct my memory of his story. > >It seems that his Cray Blitz program had a version written in fortran which his >team used for testing since getting Cray time was difficult. The fortran >version searched (IIRC) 5-7 ply. He noticed that his program tended to push >pawns a lot, and since it didn't have much of a search this would create >weaknesses, especially "pawn holes". So he added a term to the evaluator to >effectively restrict these moves and the program clearly played better. > >Then an ACM CC championship came up and they got use of a Cray, which would >increase their search depth to 8-9 ply. But the new version played badly and >lost an early game to a weak opponent. Cray Blitz wasn't using its pawns! Bob >removed the new evaluation term (I think he just added an instruction to jump >around the eval code) and the old Cray Blitz made a comeback, defeating Hans >Berliner's Hitech to win the Championship! But from seeing Cray Blitz's weak >play in earlier games Berliner became suspicious and demanded an investigation. >This was done later using another Cray to the satisfaction of then ICCA >President Levy, if not Berliner. > >So yes, an evaluation term that looks good at one depth can bring you grief with >deeper searching. > >Also IIRC, Hsu wrote that adding a term for "the initiative" hurt Deep Blue in >testing. A good search already shows you what any initiative can do. I disagree. Hsu probably did not evaluate initiative correctly. A good search cannot show you everything. There are leaf positions that you need to evaluate. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.