Author: Bob Durrett
Date: 09:53:38 01/20/04
Go up one level in this thread
The "shredder 8 and weird PVs?" thread is one of the more informative threads I've seen here in quite awhile. As a reminder for what everybody already knows: When a chess engine is given the task of automatically analyzing a game after the fact, it often produces sequences of moves which somewhat resemble PVs but are not. The difference is that each move in each such sequence is analyzed as if it were a new move. [I'm not sure what's happening in the hash table during creation of such lines, however, and am curious.] In the final printout of this analysis [which I call "overnight analysis" since I have my engines do it overnight] there are NO PVs given. With Fritz, the PV is useful but currently the PV of Shredder is just the opposite of being useful. That's a pity, from the user's perspective, since Shredder is supposed to be the "top engine." Generally, one must recognize that each move of the Fritz PV is less than "the best choice" with decreasing credibility as one goes out in the PV. Typically, the last move in a PV has almost no credibility at all. Nevertheless, Fritz's PVs are useful to the user because they show a possible way the game might progress. This helps the user to gain some insight into the position for which the given PV applies. Generally, there is the danger that the uninitiated user might "believe" the entire PV, assuming that it is correct. If one must have a correct "PV" then an "overnight analysis" should be done. That is a good starting point for subsequent human-guided analysis. One must be wary to avoid misinterpreting PVs, assuming that they are more than what they actually are. Bob D. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ On January 20, 2004 at 12:01:59, Robert Hyatt wrote: >On January 20, 2004 at 05:10:35, Sune Fischer wrote: > >> >>>I don't follow that reasoning. As a chess player, and I have been one of >>>"those" for almost 50 years now, I care greatly about the "PV". I _always_ >>>want to know what is supposed to happen, and a PV is the only way I know of >>>getting that information. I hate analysis that says "obviously Rxh7 is the >>>best move here" without giving any analysis to show why. So from a playing >>>perspective, it is important. >> >>You have searched millions of positions and all you get is a single of a dozen >>moves. >>The only sane conclusion is that the PV gives you an infinitisimal amount of >>information, it is *eye-candy*. > >Then Grandmasters are producing tons of "eye candy" every day as they >merrily annotate games and give variations. Are you really a chess player? >IE I want analysis, not just "white is better" myself. > > >> >>It never shows you all the refutations, say there is a queen en prised on the >>board, the PV at iteration 5 shows us taking it but at ply 6 we get a completely >>new line. >>Why can't we take the queen, what's wrong with that, what did the engine see? > > >That is obtainable, if you want to know. You just step down the PV to the >point in question (you _do_ have a PV to step down, right, otherwise you would >not even know the queen is hanging except by the score maybe) and then let the >search show you the PV for the move you are questioning. > > >> >>That's what I want to know, as a chessplayer :) >> > >Me too and that is what I get by having the PV displayed. With just a best >move or best move and sometimes random moves below it being shown, I have a >harder time. Just note the people that started this thread. Obviously it >meant something to them... > > > > >>>But, as I have also said, it is _critical_ (IMHO) for debugging. I always want >>>to know _exactly_ what position the program evaluated, so that I can decide if >>>I agree with the evaluation or if it needs some tuning... And without the PV, >>>how can I possibly know what my program was evaluating??? >> >>Even when I used the pv-array it almost never reached out to the evaluated leaf >>position, so I don't think it's a very good reason in practise. >>Perhaps my hash is just too effective for that ;) > >Mine almost always goes to the leaf position. On occasion it gets cut short, >but that is the exception rather than the rule. > >> >>My pv stops at qsearch, that's the part I care about mostly. >>Qsearch has its own story to tell and terminating there avoids a lot of noise. >> >>>>Doesn't work in my experience. >>>> >>>>If the root move is bad and you want to debug then probably the PV will be junk >>>>for you as well, now what do you do? >>> >>>I reduce the depth and search again, and use _that_ PV. >> >>I had a bug a few weeks ago, a mate in 6 which was the shortest mate possible, >>suddenly turned into a mate in 4 after a fairly deep search. >> >>Somehow one of the moves weren't getting refuted as they should, that was >>obvious, but the whole PV was just nonsense starting with a wrong root move. >>I couldn't use the PV for debugging at all, and the previous iteration were >>correct! >>There's a typical bug for you to hunt down. > >I don't usually use the PV to hunt down _search_ bugs. Rather, evaluation >bugs. For the search I dump the whole tree and use the PV to examine the >important parts of that huge dump. > >> >>>>I just think it is funny how much you care about the precision in the PV printed >>>>and not so much about the precision of your SEE. >>> >>>Different animals. SEE is non-critical. 99.99% of the time when it is wrong, >>>it just slightly blows move ordering... >>> >>>The PV is non-critical to crafty when playing a game, but it is _far_ >>>from non-critical when someone is annotating a game, or analyzing a >>>game. The PV is what they want, not just a score. They want to know >>>_why_ the position is good or bad, and the PV shows that. >>> >> >>Umm, eye-candy before strength, are you serious? > >I don't agree that the PV is eye-candy... > > >> >>Well at least you will get the 100% correct analysis of the games you lose. >>:) >> >>>The PV has no such computational cost... >> >>It may be not be much, but it is not zero, I can tell you that. > >It is close enough to zero to ignore, which is good enough for me since I >do want it to be accurate. > >> >>>If you don't think it useful, then >>>why reconstruct it from the hash table? You really don't need it, as you have >>>said, so just show the best root move and stop there... >> >>I like eye-candy, I like eye-candy more when it's free. >> >>>>You could also check that the depth doesn't drop more than a ply from node to >>>>node. >>> >>>UGH. That will kill fine 70. >> >>It will kill nothing, cause the pv is non-essential. >>It might make the pv shorter but more correct. >> >>>>I think unless you demand perfection there is no problem in this method. >>> >>>That is the only issue, IMHO. If you can get it right, at no cost, vs >>>right most of the time at no cost, I'll take right. But that is only >>>because I do use it all the time for debugging... >>> >>>Apparently others here (users) are also interested in seeing it as accurate >>>as possible, based on the comments that started this thread... >> >>Programmers privilege to make these decisions :) >> >>I think most users are smart enough not to trust pv's several moves out. >>They have been searched with much less depth so they are by definition much less >>accurate. >>Using a super fancy array to collect won't help you solve this fundamental >>problem :) >> >>-S. > >In tactical positions I trust the PV to the end. IE mate in 15 is not going >to be flakey near the end. Nor is winning a pawn or piece, or promoting >something... > >So it is not as useless as you seem to think.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.