Author: Robert Hyatt
Date: 14:51:27 01/19/04
Go up one level in this thread
On January 19, 2004 at 14:04:52, Sune Fischer wrote: >On January 19, 2004 at 12:28:09, Robert Hyatt wrote: > >>>It works just fine. >>> >> >>I will say this again with authority: It works just fine _most_ of the >>time. But _not_ all of the time. I've explained why, carefully. Unless >>you have some way to realize that a table entry is a PV entry when you store >>it, so that you can say "never overwrite this entry" then you _will_ suffer >>losses. You will get short PVs. And you will get PVs with wrong moves.. >> >>There is simply nothing you can do about it, any more than you can fix the >>draw problem with hashing... > >You can say it with all the authority you want, but you can't prevent me from >testing it and making up my own mind about it. And I wouldn't want to do so. However you _will_ have problems so long as you overwrite positions. And you _will_ overwrite positions with the same hash signature, else you have no way of matching the right one... So this is going to be a problem with no solution. Whether it is a major problem or not depends on you. We already have plenty of hash table problems that we can not possibly fix, yet we still hash. Nothing says you can't choose to simply add another small problem to the mix and still be happy... > >It's not perfect, I never said it was, but it is a good compromise. I never said it was bad. I said it had a _known_ problem for which there is no solution. > >How often do you actually study the moves 12 plies out in Crafty's PV when the >lines scroll down? When I am testing, _every_ time. I want to see the position that produced the score, so that I can go look inside evaluate() to see what contributed to the number I don't like... >..yes, that's what I thought. :) I suspect you thought wrong, based on what you imply. :) I demand an accurate PV/Score so that I can debug the thing and see why that particular position produces a score that I don't think is correct. If I don't have the full and correct PV, then I am not going to be able to look at the right position to see what happened... > >>>My explanation is that once the PV has actually reached that kind of depth the >>>search has been going on for so long that old PV entries have been replaced. >> >>That is certainly the problem in any of the cases being discussed. The only >>solution is to have a table so big you _never_ overwrite, and then on top of >>that you need a replacement policy that _never_ replaces an existing entry >>even if the hash signature matches, if the existing entry is a PV position. Of >>course I know of no way to identify a PV position at the point you store it... > >Actually, it seems to work just fine doing nothing out of the ordinary :) > >>>>I now do it the correct way, backing the PV up along with the score... >>> >>>IMO the PV is a non-essential feature, having to do it "correctly" messes up the >>>code and slows down the engine. >> >>It doesn't cost .1%. Just add the code, and then see how many times you >>actually use it. Hint: only when alpha < score < beta. With PVS that >>is essentially _never_. >> >>You might think it is used all the time, but in reality it is not very >>frequent. > >Hehe, and this from the guy who allows illegal king captures in the SEE :) I'm not sure I follow the reasoning here. I _know_ what the PV backup costs. I have actually measured it carefully... So there's no guesswork at all. I'm not sure how that factors into SEE. SEE was never intended to be highly accurate. By definition it is not. > >Trust me Bob, there are many things that could make the program stronger by >being more accurate, however the PV output string isn't one of them. Trust me, if you try to debug your evaluation, you _do_ need to be able to reach the position being evaluated so you can see what is contributing the oddball scoring component that you didn't expect. That was my only point here. Having a score and an accurate PV that leads to the position that actually backed up the score is important for testing. During a game it is not important at all, as you mention... But since I want it accurately for testing, and since it doesn't cost much of anything to do it accurately all the time, I simply go that way. > >>>And? >>>All engines produce not very bright move way out in the PV. >> >>Yes, but the moves they display are the moves that led to the position that >>produced the score that was backed up. > >Unless you get a hash cutoff, that's bad or is it? >Ironic really. :) It happens. And I can't debug those positions. But that is not that common when testing on a fixed position, as opposed to a pre-filled hash table during a game... > >>Using the HT to produce the PV doesn't >>guarantee that. > >There is a small chance another PV can overwritte the original PV entry, with >the same position and store a different move. >However, if you think about that the chance is not very big, and if one really >cared one could assert that the depth stored was enough as well, if deeper >that's good if too shallow cut the PV. Yes, but by definition you are making the end of your PV less reliable. Which it already is, of course. But once again, I am thinking about how I develop the code. When I see a strange score in a game, I can go to that position, follow the PV with a "reada" command, and then see the final position and type "score" to see what is happening. If that doesn't help, I can re-compile with more debugging output enabled in Evaluate() and do it again seeing every component of the score as they are computed. > >>It is not about the quality of the moves. It is about the correlation of the >>moves to the score being scored. That was what Martin first pointed out. A >>piece was hanging according to the PV. According to the score all was even. > >I can't speak for Shredder, have no idea what it is doing. >Perhaps there is no other way to extract the PV. All I was doing was explaining one known problem with extracting the PV from the hash table. This isn't a new idea, by any stretch, it has been around for 20+ years at least... > >>>You can't really rely on anything but the first few moves to be good, unless it >>>ends in a mate or or tactical win. >>> >>>As long as you extract PV moves you won't see outright blunders, if you do you >>>need to debug your search :) >> >>You can definitely see outright blunders. When you probe to find the PV >>move at ply=11, the entry you find will usually be for the right position, >>but the wrong depth. That can certainly produce a blunder, such as leaving >>a mate in 1 that the q-search couldn't find. > >If it's a PV move it's a PV move, which means the search thinks it is a good >move for _both_ sides. >How can that possibly be a blunder unless your search is broken? You reach position A and due to your remaining depth you search one move and find it good. You reach the same position with a different depth and see that another move is better. You have the same position, two different best moves and drafts. You have a potential problem. A might have seen that the other move is bad. when you reach it the second time you might not... If you haven't seen this before, I should point out that you can reach position A multiple times, at the _same_ ply, in the _same_ iteration, but with different remaining depth. Because you can transpose moves and a check becomes a non-check and vice-versa... That is again the result of the hash table having no path information included, just position... > >(Again, have no idea what is going on in Shredder). Someone else pointed out that it is possible that the move was a fail-high or fail-low move. Printing out a PV on such moves is impossible and you get _lots_ of garbage most of the time. > >-S.
This page took 0.02 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.