Author: Robert Hyatt
Date: 11:16:22 11/29/00
Go up one level in this thread
On November 28, 2000 at 20:47:52, Bruce Moreland wrote: >On November 28, 2000 at 11:56:37, Robert Hyatt wrote: > >>On November 28, 2000 at 11:50:12, Dann Corbit wrote: >> >>>On November 28, 2000 at 10:30:00, Robert Hyatt wrote: >>>[snip] >>>To add a bit, here is an output from a chess engine for one of the WAC >>>positions: >>> >>>Middlegame phase. >>> 2 -173 4 525 e5c6 d6c6 >>> 2 -173 4 1232 e5c6 d6c6 >>> 3 -188 5 1569 e5c6 d6c6 f6h5 >>> 3 -187 6 4205 g3g6 ! >>> 3 -123 6 4577 g3g6 >>> 3 -122 7 6316 f6h5 ! >>> 3 -101 7 7444 f6e8 ! >>> 3 -17 7 7746 f6e8 d6e5 d4e5 d8e8 >>> 4 -17 7 8247 f6e8 d6e5 d4e5 d8e8 >>> 4 -17 8 10898 f6e8 d6e5 d4e5 d8e8 >>> 5 -12 8 11626 f6e8 d6e5 d4e5 d8e8 a1d1 >>> 5 -11 11 22518 g3g6 ! >>> 5 383 14 33800 g3g6 !! >>> 5 999996 14 34042 g3g6 d6e5 >>> 5 999996 15 34369 g3g6 d6e5 >>>Learning score: 999996 best: 36 depth:5 hash: F45FB3C8 >>> >>>Notice that it 'found' g6 at ply 3. Was it 'solved'? Obviously not. Why not? >>>Because it had no idea how good the position was. Because of this, the choice >>>was easily abandoned at later ply. Given enough time, it found the right move >>>for the right reason and stuck to it. >> >> >>That was my point. I have no idea how Bruce could interpret my comment as >>accusing the author of the program of 'cheating'. > >I take some issue with the mention of negative stuff when there is non-negative >stuff that can be discussed without detracting from the conversation. > >For example, someone posts a game, and we can talk about the game or we can >spend three days talking about whether the opponent used a computer. You have to learn to read "in context". In what I wrote, the emphasis was on "just because a strong program lost to a weak human, don't assume that the weak human is really a human." If someone posts 100 games here where a 1700 player beat strong computers on ICC, I would lazily say "they are all cheaters." And I would probably be right in 99% of the cases. Because that just doesn't happen very often. But in the thread you mention, cheating wasn't the issue. "Don't take this result too seriously because ... " _was_ the issue to me... > >Or this current case, someone posts a perfectly fine and normal analysis of a >test position. This analysis is what you see when you run Nolot #1 through any >sufficiently good program. > >It would have been fine if you had said that you thought the solution was only >partial, because it didn't produce a win score. > >That's what you did for several posts. But in this post: > >http://www.icdchess.com/forums/1/message.shtml?141562 > >... your tone changed and you started talking about programs being tuned >intentially to find solutions in specific cases where they didn't have a clue. A factual event: At an ACM event, people were presenting papers on various computer chess topics. One was on a distributed parallel search on a hypercube. His results seemed to be "too good" to my experienced eye. He didn't explain how he tested to get the results so I made the following request/statement: "Can you explain how you tested? I have seen the following things go wrong many times: (1) in a distributed approach, often when you add nodes, you also add hash memory. That skews the results. (2) many times the "best" time out of a set of runs is used. Not the average or the worst. That skews the results. I gave a few other problems as well." Should he have taken it as an accusation? He didn't. Neither did anybody else there. It turned out that (1) was his problem. But it could have been N+1 where I gave N possible explanations. It put the question "in context". I am sure that had I just asked "Can you explain how you tested" it would not have been answered reasonable because the question is vague. > >I took issue with that and only that. I don't think it's right to start casting >aspersions about people's integrity when you clearly haven't done the >groundwork. You are taking issue with a non-happening. I didn't cast aspersions around about _any_ program. I simply said X has happened before, and as a result, I always look for the "right reason" in addition to "the right move". That doesn't _begin_ to imply I think it happened in this case. It only explains why I don't take the results as presented as "solution found." To imply more is to do what the Democrats are trying to do in Florida... to "divine" how people intended to vote without a clearly marked ballot to do so. No need to try to read my mind here.... > >Anyone who saw that human vs computer game and actually looked at the moves >could have figured out that the human really was a human. If there had been any >doubt, it would have been possible to check the guy's history to see how he >played in other games, and other games showed him attempting to go for the same >kinds of thematic elements with catastrophically bad, and very evidently human, >results. If the point of the exercise was to say "he cheated" I would have applied a totally different methodology to looking at the game. The purpose was to suggest that not all such losses are what they seem to be... > >Anyone could take the time to familiarize themselves with Nolot #1 in order to >determing that the typical program will find the "answer" in a fairly short >amount of time, with a score near zero, without feeling any need to bring up the >issue of test suite cooking in response to PV's produced by a specific program. > >>I can certainly say one thing. I hope he never writes a paper for publication >>in any journal. Because often the reviews that come back ask for clarification >>or more data, and often the reviewer will give reasons why he wants the >>clarification. And most of us would _never_ take such reasons as accusation >>that _we_ did the same thing. We would take them as an explanation for why >>the reviewer felt more information was needed. > >I don't understand this but it sounds a little ad hominem, which is not a >particularly big deal. I'm not taking any of this personally, and I'm not >meaning to be taken personally. > >I'm responding with the same part of my belief structure that responds when >Kasparov accuses DB of cheating, or when someone says that Hsu is stupid, or >when someone says that Hsu is something beyond brilliant. > >I am not a person who likes to wonder about who killed JFK, and my experience >dealing with people on the net leads me to understand that people are all too >happy to blame virtually anyone for killing JFK. I haven't seen a big human vs >human event on ICC yet where someone didn't announce that the fix was in. > >I don't believe in saying that the fix is in, or that someone cheated, or that >someone killed JFK, or that aliens abducted and replaced one player with an >alien, unless there is good evidence that this has happened. Lacking good >evidence, I'd at least like to see something even remotely plausible advanced, >I'd like to at least have to stop to think about it, rather than immediately >realize that this is just another internet accusation grounded in nothing. > >I don't have a problem with someone else discussing someone's integrity, but >only when the groundwork has been laid. And in both of these cases, I don't >think that *any* groundwork was laid. > >>"I don't trust a solution that has the right move but the wrong score, because >>I have seen (a) programs tuned to choose the right move to improve their test >>result scores artificially; (b) I have seen programs later change their mind >>and not select the right move, given more time, because they didn't understand >>how good the original move was; (c) I have seen programs play the right first >>move, but veer off into a perpetual for the same reason." Was what I said. I >>see _no_ way to take that as an accusation that the programmer/program in >>question did any of those. Instead, that is the reason why _I_ am personally >>skeptical of right move wrong score solutions, period. > >You also said this: > >"There have been multiple cases of programs being specifically tuned to play >certain moves in test positions. The PV was "close" but missed the important >move that made the score _really_ improve." > >There is an implication in there that I object to, since there is no evidence >that would support the notion that Gandalf has been tuned to do anything here. >It's just doing what all good programs do here. I bet Crafty does it too. > >That is the only problem I have with this. > >bruce
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.