Author: martin fierz
Date: 08:59:16 03/02/04
Go up one level in this thread
On March 02, 2004 at 11:38:35, Stephen Ham wrote: >On March 02, 2004 at 10:29:40, martin fierz wrote: > >>On March 02, 2004 at 10:08:15, Stephen Ham wrote: >> >>>On March 02, 2004 at 05:19:52, martin fierz wrote: >>> >>>>On March 01, 2004 at 14:40:29, Stephen Ham wrote: >>>> >>>>>On March 01, 2004 at 14:06:15, William Penn wrote: >>>>> >>>>>>Due to a unique bug in Shredder 7 and Shredder 8, its analysis is unreliable. >>>>>>Only the first move can be trusted. Forget the rest! Sometimes it's OK, but >>>>>>often it's garbage, or somewhere in-between. If you want reliable analysis, >>>>>>choose a different engine (not Shredder). >>>>>>WP >>>>> >>>>>Dear William, >>>>> >>>>>I suspect that you misread my post. Yes, in Shredder 7 & 8, only the PV is >>>>>reliable. This is true for some other engines too. For example, I noticed this >>>>>in my matches with Friz 6a and Nimzo 7.32. >>>>>(http://www.correspondencechess.com/campbell/ham/ham.htm). But the a-pawn losing >>>>>move that was second in move-ordering was the PV of the second line of the move >>>>>ordering sequence. >>>>> >>>>>All the best, >>>>>Stephen >>>> >>>>hi stephen >>>> >>>>while i understand what you want to say, you are not saying it correctly. >>>>the PV is the whole line given by shredder. the first move of the PV is always >>>>correct, the entire rest of the PV *can* be garbage but must not. >>>> >>>>so your statemtent "only the PV is reliable" is totally wrong, but i think >>>>that's because you use PV in the sense of "first move of PV" - but since you're >>>>the only one who does this, it will lead to confusion :-) >>>> >>>>cheers >>>> martin >>> >>>Hi Martin, >>> >>>Sorry for my poor communications. My ignorance of chess engine vernacular is >>>showing. Since I'm in finance professionally, PV has a meaning for me which I >>>assumed to be similar to the data output in chess engines. Therefore it may be >>>best to offer an illustration of what I meant. >>> >>> >>>27 Re1 blah blah blah blah blah uzw... +1.09 16-ply depth >>> >>>What I meant by PV was merely the 1-ply that is represented by 27 Re1, since the >>>remaining thread in S7 and S8 is highly suspect. >> >>right, the first move of the PV is correct, the rest of the PV is suspect - in >>shredder, but also in some other programs; those which take the PV from the >>hashtable. PV stands for "principal variation", so it's the entire variation >>given above, Re1 including the whole blahblah. >> >> >>>However, after move ordering through 16-ply depth made 27 Re1 the prime >>>candidate to be played, it then picked 27 a3 to be the next move analyzed in the >>>move ordering sequence. It's my understanding that this move ordering process >>>allocates greater time for analysis of the best candidates. >> >>well.... i don't think so! it depends a bit on what you're doing. if you are >>using shredder in multi-variation mode, then this is correct. i.e. if you have >>it display the best N moves, then the best N moves are ordered by value, and it >>computes an exact value for them. keep in mind that normally, programs only >>compute a value for the best move, and prove that all others are worse. >>therefore, root move ordering doesn't usually mean that the second move in the >>list is the second best; simply because the program has no idea which of the >>remaining N-1 moves has the second best value - it only knows that all of them >>have a lower (or equal) value than the best move. >>one common technique is to order the remaining N-1 moves according to the number >>of nodes it took to "refute" them, i.e. to prove that they are not better than >>the current best move. the idea being that you need very few nodes to prove that >>a really crappy move is in fact crappy. but still, this doesn't order moves >>according to value. >> >>so, if you were not using "display N best moves" with N>1, the 27.a3 move is not >>second best just because it's searched as second move. >> >>>So while S8 still displayed 27 Re1 as the best move, its analysis next switched >>>to 27 a3 blah blah blah blah blah uzw. It's the 27 a3 move that just drops a >>>pawn, so I was shocked to see it placed second in the move ordering sequence. >> >>for this not to shock you, you must understand that it doesn't really matter as >>long as Re1 is really the best move. then you must search all remaining moves, >>and the order in which you search them is irrelevant. >>the only time this gets relevant is when Re1 is in fact not the best move, and >>your time is running out, and you only manage to search a couple of the >>remaining moves after searching Re1, and you would have found that 27. XY is >>better than Re1 but unfortunately you searched 27. a3 and other crap first and >>wasted your time, and didn't find the better move when you decided it was time >>to play now. >> >>my personal experience with computers and deep analysis is that all automated >>analysis is inferior to human-guided analysis. let the machine think a bit, in N >>best moves mode, see whether you agree and move forward along a sensible line. >>the "deep position analysis" feature of the fritz family mimics this behavior, >>but i don't like it too much. i have seen fritz look at really weird moves in >>it's deep position analysis feature, which i think you could confidently skip as >>a human; so i would do it all interactively. >> >>cheers >> martin > >Dear Martin, > >Thanks for your highly educational post. I always assumed that PV meant "present >value", as it does in Finance (my mistake). So I mistook PV to be akin to the >"root" of the analysis tree. > >Also, I've learned that chess engines are more likely to find the strongest move >only when displaying the PV. Therefore I only had it do that (rather than having >it display the top 3-lines, for example). they of course will still give the same result, but they need more time to arrive at it. in this sense your "more likely" is correct, because they don't have to spend time to find the real value of moves 2...N, they can search a bit faster. i don't know how much the speedup is though from 1 move to say 3. >As for move ordering, I've noticed that (all things being equal) the second move >in the ordering sequence virually always gets more calculation time than the >10th, which in turn gets more than the 20th. yes, that is correct.... > So I assumed that this was the >engines way to devote more calculation time to meaningful candidates and less >time to moves that are probably weak. But you're now telling me that this is >incorrect, Martin. So then, why are the lines placed closest to the PV in the >move ordering process given generally more time? ...it's not that they are given more time: as i tried to explain in the last post, once you have searched the first, and probably best, move, you will order the remaining moves in such a way that the one which needs most nodes to be refuted will come second, and so on (of course you just measure this at depth X and then use this data at depth X+1 for the ordering, you can't know in advance). since nodes ~ time, this results in the behavior you observe: the second move is considered for a longer time than those toward the end of the list. but this doesn't really mean that the engine is conciously searching these moves deeper than the final moves (for example by reducing the search depth on the final moves), it's just that it needs longer to refute these moves (again, refute meaning to show that they are not better than the 1st move). the whole point for this root move ordering scheme is that moves which are obviously ridiculous are easily refuted (meaning with few nodes and little time). e.g. if you hang your queen with a move, it is usually easy to refute. moves which are not so easily refuted are mostly not quite as ridiculous. but finally, this type of move ordering obviously cannot succeed with high accuracy: think about a position, where you have a couple of candidate moves: your favorite A, two others B and C. B is a move leading to spectacular complications. C is a quiet move. we humans will always need much more time for B than for C - and yet it is quite possible that B is worse than C. for computers, things are similar. e.g. that a3 move which drops a pawn might lead to some kind of complication which makes shredder need lots of time to refute it. >For the record, as a CC player, I find my own moves and don't use engines to >generate my moves. Therefore I don't have much interest in DPA other than as a >curiosity. Instead, my enjoyment comes in subsequently checking my games against >engines to see which are most likely to find my moves, or to show me my errors. >I tend to play in a rather technical/positional style, so engines often have >difficulty finding moves that are related to long-range planning, versus winning >pawns, etc. This is time consuming so I admit to having only conducted a couple >dozen tests. An example of these tests is seen in my review of S7 at >ChessCafe.com. So far, all of my tests were conducted in Infinite Analysis mode. >Now I'm wondering if I should have instead conducted my analysis in DPA mode. >Any thoughts, Martin? Thanks again for your valuable input. again, neither is satisfactory. the problem is the exponential growth of the game tree, which more or less limits the usefulness of a deep search from the root position. there are many highly tactical positions which the computers don't evaluate well because of this. e.g. i recently played a tournament where the guy at the board beside me had the opportunity of saccing both his rooks with Qxa1+ K moves Qxh1 and getting a strong attack. he didn't do it, and of course we analyzed it later. i went home to check it out, and if you start any program in the original position, it won't choose to sac the material. play the first move of the combination, then take both rooks, and you are already 5 ply into the game tree. play the next rather obvious move and the forced response, and you are 7 ply deep. do a deep search from there. you have won a lot. this type of analysis is obvious to the human expert. for the computers, if you do the standard 16ply search in the root position, they look at all other stuff that you don't really want to know. use deep position analysis, and they might not even list that first sac in their first couple of moves, and won't find it at all. the whole concept of "critical variation" doesn't exist for computers. DPA will also look at lots of other moves than the "critical move". for me it's obvious that you have to look at this sac, so i do it. DPA just generates the first X best moves and looks at all of these. i'll try to find that position again (it's a year ago or so since i saw it), and if i find it, i'll try in DPA mode, and with a deep search, and post it here. if i remember right, it took me about an hour to prove that the sac wins, combining my human insight with the computer's power. i'll see how long it takes with these other methods. cheers martin
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.