Author: Robert Hyatt
Date: 18:30:36 01/19/04
Go up one level in this thread
On January 19, 2004 at 19:34:48, Sune Fischer wrote:
>On January 19, 2004 at 17:51:27, Robert Hyatt wrote:
>
>>And I wouldn't want to do so. However you _will_ have problems so long as
>>you overwrite positions. And you _will_ overwrite positions with the same
>>hash signature, else you have no way of matching the right one... So this
>>is going to be a problem with no solution. Whether it is a major problem or
>>not depends on you. We already have plenty of hash table problems that we
>>can not possibly fix, yet we still hash. Nothing says you can't choose to
>>simply add another small problem to the mix and still be happy...
>
>There simply is no problem as I see it, cause I have no use for the PV.
>
>The PV is just a little fun to see what the engine thought was the best line, as
>a debug tool it might be helpful but otherwise it's just a bit of fun.
I don't follow that reasoning. As a chess player, and I have been one of
"those" for almost 50 years now, I care greatly about the "PV". I _always_
want to know what is supposed to happen, and a PV is the only way I know of
getting that information. I hate analysis that says "obviously Rxh7 is the
best move here" without giving any analysis to show why. So from a playing
perspective, it is important.
But, as I have also said, it is _critical_ (IMHO) for debugging. I always want
to know _exactly_ what position the program evaluated, so that I can decide if
I agree with the evaluation or if it needs some tuning... And without the PV,
how can I possibly know what my program was evaluating???
>
>If I really want to analyze a position with an engine I don't let it crunch for
>20 hours on a position to study the PV, what I do is to play out the lines I
>find interesting and see if the engine can find improvements or tactical
>refutations.
>
>>>..yes, that's what I thought. :)
>>
>>I suspect you thought wrong, based on what you imply. :)
>>
>>I demand an accurate PV/Score so that I can debug the thing and see why that
>>particular position produces a score that I don't think is correct. If I
>>don't have the full and correct PV, then I am not going to be able to look
>>at the right position to see what happened...
>>
>
>Doesn't work in my experience.
>
>If the root move is bad and you want to debug then probably the PV will be junk
>for you as well, now what do you do?
I reduce the depth and search again, and use _that_ PV.
>
>Analyzing the wrong PV is doesn't get you anywhere, the failure is in one of
the
>refutations, a move which should have FH must have FL or vice versa.
You are not thinking about the right kind of analysis. I am talking
about analyzing a _position_ reached in the game. If I am not happy
with the PV, because it doesn't extend in the right place, that is yet
another important reason for having a good PV to look at, so that you
can _confirm_ that the program saw the tactical issue by searching the
right things. But when Crafty is playing a game, and I see a -.7 when I
think white is better, I make a note, and back up to that move later,
play down the pv, type "score" and see what is going on...
>
>>>Hehe, and this from the guy who allows illegal king captures in the SEE :)
>>
>>I'm not sure I follow the reasoning here. I _know_ what the PV backup costs.
>>I have actually measured it carefully... So there's no guesswork at all. I'm
>>not sure how that factors into SEE. SEE was never intended to be highly
>>accurate. By definition it is not.
>
>I just think it is funny how much you care about the precision in the PV printed
>and not so much about the precision of your SEE.
Different animals. SEE is non-critical. 99.99% of the time when it is wrong,
it just slightly blows move ordering...
The PV is non-critical to crafty when playing a game, but it is _far_
from non-critical when someone is annotating a game, or analyzing a
game. The PV is what they want, not just a score. They want to know
_why_ the position is good or bad, and the PV shows that.
>
>I have the complete opposite priority :)
>
>I don't care if my PV every 20th time has an inaccuracy at ply 11, I do care if
>my SEE is inaccurate every 20th time (more than it needs to be)!
>
>>>Trust me Bob, there are many things that could make the program stronger by
>>>being more accurate, however the PV output string isn't one of them.
>>
>>Trust me, if you try to debug your evaluation, you _do_ need to be able to
>>reach the position being evaluated so you can see what is contributing the
>>oddball scoring component that you didn't expect. That was my only point
>>here. Having a score and an accurate PV that leads to the position that
>>actually backed up the score is important for testing. During a game it
>>is not important at all, as you mention... But since I want it accurately
>>for testing, and since it doesn't cost much of anything to do it accurately
>>all the time, I simply go that way.
>
>I will quote you from memory:
>
>"if you add an if here and an if there you can go from 2 Mnps to 1 Mnps in no
>time"
>
>;-)
>
The PV has no such computational cost... If you don't think it useful, then
why reconstruct it from the hash table? You really don't need it, as you have
said, so just show the best root move and stop there...
>>Yes, but by definition you are making the end of your PV less reliable. Which
>>it already is, of course. But once again, I am thinking about how I develop
>>the code. When I see a strange score in a game, I can go to that position,
>>follow the PV with a "reada" command, and then see the final position and
>>type "score" to see what is happening. If that doesn't help, I can re-compile
>>with more debugging output enabled in Evaluate() and do it again seeing every
>>component of the score as they are computed.
>
>I believe I can make it completely reliable if I wanted to use it for debugging.
I can guarantee you that you can't. This is a simple matter of hash
replacement, and there is no way to fix this without breaking hashing
totally. IE fix the draw score problem and you could fix this problem.
But the draw score problem is not fixable without just tossing hashing
out.
>
>The important thing is not to clutter the search with non-search related things,
>not just for the speed but more for the readability of everything.
>
>Just my personal preference of course. YMMV
This doesn't clutter the search...
memcpy(&tree->pv[ply - 1].path[ply], &tree->pv[ply].path[ply],
(tree->pv[ply].pathl - ply + 1) * sizeof(int));
I don't see how that is making anything more complex. First, it is hardly
ever executed, and second, it is only one line of code that could be written
as a macro to further hide the messiness...
>
>>>>It is not about the quality of the moves. It is about the correlation of the
>>>>moves to the score being scored. That was what Martin first pointed out. A
>>>>piece was hanging according to the PV. According to the score all was even.
>>>
>>>I can't speak for Shredder, have no idea what it is doing.
>>>Perhaps there is no other way to extract the PV.
>>
>>All I was doing was explaining one known problem with extracting the PV from
>>the hash table. This isn't a new idea, by any stretch, it has been around for
>>20+ years at least...
>
>And I thank you for that.
>
>I feel like I am on top of things here though, damage control, risk factor
>everything weighed and balanced I am confident in my choice. :)
>
>>>If it's a PV move it's a PV move, which means the search thinks it is a good
>>>move for _both_ sides.
>>>How can that possibly be a blunder unless your search is broken?
>>
>>You reach position A and due to your remaining depth you search one move and
>>find it good. You reach the same position with a different depth and see that
>>another move is better. You have the same position, two different best moves
>>and drafts. You have a potential problem. A might have seen that the other
>>move is bad. when you reach it the second time you might not...
>>If you haven't seen this before, I should point out that you can reach
>>position A multiple times, at the _same_ ply, in the _same_ iteration, but
>>with different remaining depth. Because you can transpose moves and a check
>>becomes a non-check and vice-versa... That is again the result of the hash
>>table having no path information included, just position...
>
>There are generally so few pv nodes that this isn't a big problem.
>Usually the entry gets smashed by an upper or lower store.
>
>You could also check that the depth doesn't drop more than a ply from node to
>node.
UGH. That will kill fine 70.
>
>I think unless you demand perfection there is no problem in this method.
That is the only issue, IMHO. If you can get it right, at no cost, vs
right most of the time at no cost, I'll take right. But that is only
because I do use it all the time for debugging...
Apparently others here (users) are also interested in seeing it as accurate
as possible, based on the comments that started this thread...
>
>>>(Again, have no idea what is going on in Shredder).
>>
>>Someone else pointed out that it is possible that the move was a fail-high
>>or fail-low move. Printing out a PV on such moves is impossible and you get
>>_lots_ of garbage most of the time.
>
>Yes a null window search couldn't have produced any pv moves, but what if
>Shredder isn't doing PVS but something entirely different?
>
>Perhaps the PV is deliberately scrambled. It is anyones guess.
>
>-S.
This page took 0.02 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.