Author: Robert Hyatt
Date: 11:38:33 11/28/00
Go up one level in this thread
On November 28, 2000 at 14:12:53, Peter McKenzie wrote: >On November 28, 2000 at 13:44:15, Robert Hyatt wrote: > >>On November 28, 2000 at 12:47:00, Mogens Larsen wrote: >> >>>On November 28, 2000 at 12:12:12, Mogens Larsen wrote: >>> >>>>On November 28, 2000 at 11:50:12, Dann Corbit wrote: >>>> >>>>>On November 28, 2000 at 10:30:00, Robert Hyatt wrote: >>>>>[snip] >>>>>To add a bit, here is an output from a chess engine for one of the WAC >>>>>positions: >>>>> >>>>>Middlegame phase. >>>>> 2 -173 4 525 e5c6 d6c6 >>>>> 2 -173 4 1232 e5c6 d6c6 >>>>> 3 -188 5 1569 e5c6 d6c6 f6h5 >>>>> 3 -187 6 4205 g3g6 ! >>>>> 3 -123 6 4577 g3g6 >>>>> 3 -122 7 6316 f6h5 ! >>>>> 3 -101 7 7444 f6e8 ! >>>>> 3 -17 7 7746 f6e8 d6e5 d4e5 d8e8 >>>>> 4 -17 7 8247 f6e8 d6e5 d4e5 d8e8 >>>>> 4 -17 8 10898 f6e8 d6e5 d4e5 d8e8 >>>>> 5 -12 8 11626 f6e8 d6e5 d4e5 d8e8 a1d1 >>>>> 5 -11 11 22518 g3g6 ! >>>>> 5 383 14 33800 g3g6 !! >>>>> 5 999996 14 34042 g3g6 d6e5 >>>>> 5 999996 15 34369 g3g6 d6e5 >>>>>Learning score: 999996 best: 36 depth:5 hash: F45FB3C8 >>>>> >>>>>Notice that it 'found' g6 at ply 3. Was it 'solved'? Obviously not. Why not? >>>>>Because it had no idea how good the position was. Because of this, the choice >>>>>was easily abandoned at later ply. Given enough time, it found the right move >>>>>for the right reason and stuck to it. >>> >>>That is an obvious case but you're oversimplifying the question at hand. First >>>of all it found the move at ply 5 for the same reasons as at ply 3, ie. better >>>evaluation than other possibilities. You conclude that it found the right move >>>because the evaluation is improving (rapidly), because you know that it should. >>> >>>The fact that the score explodes is irrelevant as it depends on the position, >>>eg. finding a mating sequence or finding a slightly better move. So going from >>>-167 to 11 slowly as an example is just as valid as -11 to 383 rapidly. It >>>depends on the nature of the position and the depth of the correct variation. >>>Guessing the correct first move in a long variation with a high score is even >>>more suspicious IMO. The evaluation score isn't a right reason by itself. >>> >>>My interpretation of right reason is an improvement in score as the PV >>>approaches the "correct" variation. The Gandalf case doesn't justify this "right >>>reason" conclusively, but it's very close. I find my interpretation easy to >>>understand, independent of position type and free of suspicious speculation. >>> >>>Mogens. >> >> >>I don't have any "suspicious speculation" in this case. Only a strong feeling >>that "right move, wrong reason" is not convincing me of very much. IE on ICC >>the other night we had a long discussion about a move Crafty had played. The >>GM (Don't remember who it was not, but not Roman) said "Rd4 was beautiful... >>I am very impressed that the program saw it as it led to a crushing position >>for it." I looked at the log and told the group discussing this "The score was >>very bad for it at that point... it actually thought that move was best, but >>that it was losing the game... until it failed high on the 'pondering search' >>after playing it." We all agreed that it was just "lucky" there that it found >>a move that turned into a win, even though it had no idea when it played it how >>it would turn out... > >Remember the saying that goes something like, 'good players tend to get lucky' ? > >I think thats what this whole thing boils down to. If you (a program or a >human) have a few clues about what is going on, then you are likely to play good >moves. And as Larsen said: 'Good moves win chess games, not good positions'. > >I wouldn't get too carried away with this 'right reason' stuff, I think it over >simplifies things. Counterpoint would say that "right move and nothing else" also over-simplifies things. Because the question is "do you play the right move for the wrong reason and later discover it is winning more often than when you play the right move for the wrong reason and later discover the bottom drops out?" We have all (most everybody anyway) always counted a test position right if the key move was found. And that is not a bad way of measuring very large test suite results (IE WAC, ECM, etc...) because the "wrong reason" has to catch up with you statistically. But on a very small (Nolot) suite, or even on a single position from a small suite, I think a "microscope" is a good analysis tool. I personally don't feel very "safe" if my program is doing something good for the completely wrong reason(s) it found... yes, I like to see it do the right thing, period. But those "wrong reason" cases cause me to remember that for every right move, wrong reason, there will also be wrong move, wrong reason cases as well. > >> >>That was my point. Yes it played the right move. No it didn't understand >>why. It played it fully expecting to lose. It could have played any of >>_several_ moves and _still_ expected to lose. It just _happened_ to pick the >>right one. In this case, for a reason (weak opponent pawn) that had _nothing_ >>to do with how the game actually progressed and was won by that move. >> >>I chalked that one up to heads or tails and heads came up _that_ time. Next >>time it could well be tails and it goes down in brilliant flames...
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.