Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: The failure of validation with DEEP BLUE 2

Author: Robert Hyatt

Date: 12:39:11 07/18/02

Go up one level in this thread


On July 18, 2002 at 14:46:50, Rolf Tueschen wrote:

>On July 18, 2002 at 13:37:26, Robert Hyatt wrote:
>
>>On July 18, 2002 at 13:17:29, Rolf Tueschen wrote:
>>
>>>On July 18, 2002 at 12:10:03, Robert Hyatt wrote:
>>>
>>>>On July 18, 2002 at 07:57:15, Rolf Tueschen wrote:
>>>>
>>>>>On July 18, 2002 at 01:00:11, Robert Hyatt wrote:
>>>>>
>>>>>>On July 17, 2002 at 17:30:01, Rolf Tueschen wrote:
>>>>>>
>>>>>>>On July 17, 2002 at 09:23:53, Robert Hyatt wrote:
>>>>>>>
>>>>>>>>On July 17, 2002 at 05:24:50, Rolf Tueschen wrote:
>>>>>>>>
>>>>>>>>>On July 17, 2002 at 00:03:24, K. Burcham wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>In game 2,  Kasparov thought that there was human interference with this line.
>>>>>>>>>>He requested the logs to see for himself that these two moves were actually in
>>>>>>>>>>the Deep Blue eval.
>>>>>>>>>>Kasparov did not think any program would play 36.axb5 avoiding 36.Qb6 or the
>>>>>>>>>>move 37.Be4.
>>>>>>>>>>
>>>>>>>>>>But it seems that todays programs will accomplish what Deep Blue was trying to
>>>>>>>>>>do in the game.
>>>>>>>>>>Deep Blue opened up the a file and blocked Kasparov's play with 37.Be4, limiting
>>>>>>>>>>blacks mobility.
>>>>>>>>>>
>>>>>>>>>>Below in the examples you will see that two of todays strongest programs will
>>>>>>>>>>also accomplish this same objective. Both Fritz7 and Chess Tiger 14.0 will open
>>>>>>>>>>the a file and control the a file. also both Fritz7 and Chess Tiger 14.0 will
>>>>>>>>>>play Be4 limiting Kasparov's mobility with black.
>>>>>>>>>>
>>>>>>>>>>All three programs, Deep Blue, Chess Tiger 14.0 and Fritz7 put the
>>>>>>>>>>squeeze on black, blocked with the Be4 move, opened the a file, threatened to
>>>>>>>>>>capture blacks bishop, forced black to protect the loss of pawns, etc.
>>>>>>>>>>
>>>>>>>>>>I started each program after Kasparov's move 35...Bxd6.
>>>>>>>>>>after  each program analyzed for several hours, I took the line from each
>>>>>>>>>>program and played it through to the position after blacks move 40.
>>>>>>>>>>This way we can cover both controversial moves>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>What is the error in such experiments?
>>>>>>>>>
>>>>>>>>>Answer: You can't _prove_ something as authentic with repetitions on different
>>>>>>>>>machines built-up _after_ the event. History of CC has shown that we could never
>>>>>>>>>exclude special preps right on to the point. Therefore, logically, we cannot
>>>>>>>>>accept such "proofs".
>>>>>>>>
>>>>>>>>You _can_ disprove Kasparov's main "claim".  That "no computer would play ..."
>>>>>>>>By demonstrating that at _least_ one computer _would_ play that move, his
>>>>>>>>statement is disproven for all time.  And the rest of his claim can therefore
>>>>>>>>be taken with a mountain of salt.
>>>>>>>
>>>>>>>Not that I _ever_ was or would be your teacher of English resp. American
>>>>>>>English, but I must insist that Kasparov did _never_ say that no computer would
>>>>>>>_E V E R_ play these moves.
>>>>>>
>>>>>>First, let's get the quote right.  He did say "no computer _could_ play this
>>>>>>move.."
>>>>>>
>>>>>>
>>>>>>> What he meant was at the time being and he was
>>>>>>
>>>>>>
>>>>>>And here you go on a tangent.  Not saying what he _said_ but what he
>>>>>>"meant"...  judge says "inadmissable, that is a conclusion, not a fact."
>>>>>
>>>>>No, I do not go on a tangent, because always we must interpret language. It is a
>>>>>tour de tangent if you pretend to be able to prove exactly what a certain phrase
>>>>>means out of itself. You are right, he said it. But what it means, what he said,
>>>>>is a question of interpretation. Now, you have a _little_ problem.
>>>>
>>>>I don't have any problem at all.  "no computer could play that move" is not
>>>>open to "interpretation".  Each word has a precise meaning.  The
>>>>"interpretation" that he later emphasized was "no computer could play such
>>>>a move, so DB had human help in the background, somehow."
>>>>
>>>
>>>No computer - does this mean for you that no comp _ever_?
>>
>>"no computer" means "no computer".  Since he didn't say "No computer for
>>the next ten years" I take that to be absolute.
>
>A big mistake!
>

A big mistake to assume he uses Webster's dictionary to define his
words?  Or do we have to assume he has a Bill Clinton dictionary with
an alternative definition of "is" or "no".???



>>
>>
>>>And then, just below you give us the main reason for the reason why Kasparov
>>>politely asked after game two. I'll continue the argument down there.
>>>
>>>
>>>>
>>>>
>>>>
>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>>talking about DEEP BLUE 2 in the first place. Now back to the many tries to find
>>>>>>>the moves with our commercial or amateur programs. Here I must insist.
>>>>>>>Afterwards (with so many possibilities to interfere) you can never prove that
>>>>>>>such progs could have found the moves at the time being. But with respect to the
>>>>>>>position of Ed Schroeder I would say that even if some alien prog could have
>>>>>>>found the  moves, we are talking about DEEP BLUE 2 and we always were. Now - the
>>>>>>>deconstruction was the worst thing that could have happened. Because now we
>>>>>>>don't have any possibilities at all to corroborate or reject Kasparov's thesis.
>>>>>>>This alone was and still is the biggest violation of all ethics.
>>>>>>>
>>>>>>>Just another thought to think about.
>>>>>>
>>>>>>See my comment to Amir.  DB2 searched Qb6 and _every_ iteration the score
>>>>>>dropped.  Until it dropped all the way to +48 and the program decided that
>>>>>>axb5 was better by about 1/10th of a pawn.  No mystery.  No magic.  Just
>>>>>>the opinion of the program, right or wrong.
>>>>>>
>>>>>>It's _all_ in the logs, if you just look.
>>>>>
>>>>>You know the answer! You yourself explained what "logs" really could mean at
>>>>>best. I don't want to re-open that can but I want to mention that after all what
>>>>>happened IBM and DB2 team _still_ are in a state of emergency concerning
>>>>>"explanations" and evidence. Ok, you can hold this open until the final
>>>>>solution, but for me, excuse me, this is not ok. You are in a double bind. DB2
>>>>>team has to offer evidence, in special after the deliberate deconstruction of
>>>>>the machine. Now we'll never be able to repeat some thought processes. Bad luck
>>>>>for IBM/ DB2 team. They didn't pass the doping control. Their World Record can't
>>>>>be accepted as valid.
>>>>>
>>>>
>>>>It can by those of us that know them.  Otherwise _no_ computer vs human
>>>>game/match can _ever_ be trusted.  As I have said many times, there is _no_
>>>>way to be 100% certain there is no outside intervention...
>>>
>>>Ok. Thanks again. But this is the main reason why I am on Kasparov's side and
>>>_not_ on DB2 team's side. No matter of how many contracts they signed. If you
>>>cannot prove the output, then you'd have to deal the question with care. And you
>>>won't offend the human champion - or you want to do harm and win the show on a
>>>privat backyard (of IBM). I wrote about fair play and all that. Bob, it's here
>>>with you as in the clubs where you analyse games. There are always some people
>>>who shout a shout some two or three moves variations and then an expert explains
>>>patiently that this doesn't work because of such and such. You know, your two or
>>>three moves are absolutely ok. But in the context they are false. So, we have
>>>the eternal problem of Yes and No at the same time. At first we should find a
>>>mutual understanding on such interesting results of logic. Yes and No at the
>>>same time, simply because levels or perspectives are mixed up.
>>>
>>
>>
>>The problem is this.  There have been _hundreds_ of computer vs human chess
>>matches over the past 30 years.  _hundreds_.  Just as there have been hundreds
>>of automobile races.  But if a race is being held tomorrow, the "contract"
>>will most likely stipulate the same things that a contract for a race last
>>week stipulated.  Because that _worked_.  The "cheating" issue has been an
>>issue since "the Turk" was done hundreds of years ago.  It will remain an
>>issue for all time.  Because some things simply can not be controlled.
>>
>>Therefore, there is little point in arguing about things that are beyond your
>>control, or anybody elses...
>
>This is a mistake. It's not about control, it's about research after the event.
>In the 30 years before such a research was not necessary. For 1997 it wasn't
>necessary because of Kasparov. He just had asked questions. The team was it. At
>first GM Benjamin deceived everyone with his promise that he would give the
>pre-games after the event. Then the team acted impolitely. And then the machine
>was deconstructed! Three events I know of.

And of course the logs are public and have been for years now.  I have all
6 of them here for example.



>
>
>>
>>>At first they denied him the prints of the output and then after they "won" the
>>>show event, they deconstructed the whole machine! No, you won't be able to
>>>defend or help them out of the mess in a life-time.
>>
>>
>>They _should_ have denied him the logs _during_ the match.  He would hardly
>>sit down with the DB team and tell them where DB had made mistakes, so that they
>>might fix it.  Why should they tell _him_ what DB was seeing so that he might
>>find a problem and exploit it?
>
>You don't answer to the point. I didn't say what they should have done exactly.
>A smart team has several choices! I would have known what to do...

Me too.  I would have said "no".  You don't tell us what you are thinking about
during a game.  Neither should Deep Blue tell you what it was thinking about
during the game by letting you see the logs.

end of that story before it even gets started...


>
>>  Have you _ever_ seen (say) a WC match where
>>the two GMs sit down after each game and say "had you done this, I would have
>>done this.  Had you tried this opening move I had this trap ready to spring
>>on you..."
>
>Bob! I didn't talk about opening moves or prep. The middlegame was the point. If
>I did see such behavior? Yes! 1972! Wch!!! But I didn't start the WC debate.
>Then all the show acts in Mainz or Frankfurt.

Of course, you do _not_ see such behavior _during_ a match...  And you do
realize that DB's logs could show multiple book moves that _might_ be played,
giving away information no GM would _ever_ reveal until after-the-fact...

IE Crafty's logs certainly reveal all the moves, the various probabilities
for playing them, learned results, etc...

Should a GM have access to that during a match, when he doesn't tell me _his_
move probabilities?  I don't think so...

>
>
>>
>>Of course you haven't...  and it shouldn't have happened in NYC either.
>
>Show events! Please do not support the upcoming delusion in computerchess.
>

What "delusion"???  Kasparov lost.  So what?  Would he lose again?  Who
knows.  If he went into match 3 as psyched as he was 1/2 wat thru match
2, he would have probably lost every game...


>
>>>>>>
>>>>>>>
>>>>>>>We all know that DB 2 did not use the typical nullmove strategies of today's
>>>>>>>chessprogs. It would be the least what you and your collegues could do, that you
>>>>>>>elaborate what this could mean for the question of Qxb6 vs. axb5. Just for the
>>>>>>>sake of our own class of debate. If we could show that DEEP BLUE 2 would have
>>>>>>>been a much more difficult task to reject Qb6
>>>>>>
>>>>>>
>>>>>>Just look at their log file, and the variation they produced (note that this
>>>>>>is never the full variation on deep blue since the hardware does not supply
>>>>>>a PV, meaning you only get the software PV and since it is from the hash
>>>>>>table it is not completely reliable nor always complete.
>>>>>>
>>>>>>The log shows why they rejected Qb6 quite clearly..
>>>>>
>>>>>Maybe, maybe. Without doping control, not valid unfortunately.
>>>>
>>>>
>>>>Based on that logic, _nothing_ dealing with computers would be valid, because
>>>>ensuring no outside influence would be impossible in the context of the match
>>>>between a human and a computer.
>>>
>>>This is a typical mistake for beginners of scientifical methodology. Forget
>>>about all the outside influences, if the machine had not been deconstructed, we
>>>could check it out now.
>>
>>
>>How?  If they had had a human force a specific move, you don't think they
>>could do a quick tweak to the evaluation function or search rules to make
>>the machine _now_ play that move on its own?  So you could prove _exactly_
>>what after the fact?  Answer:  zilch
>
>(Sigh.) That was exactly the reason for the research. Because the couldn't have
>changed basics.

I have no idea what you are talking about nor implying here.  If I forced
Crafty to make a specific move because a GM said "you must play this now" I
could certainly, given a few minutes, tweak the program so that if someone
asked "can you show us that position again and let us see why it thought that
was good" I would be able to produce that move _without_ intervention the
secomd time around.  And there would be no prevention for such...



>
>>
>>So once again, this discussion leads to nowhere...
>>
>
>It's called subtle ! The results of our discussion are already there.
>
>>
>>>If we had some games or variations from the prior-match
>>>practice, we would also have much more insight, of course in addition with the
>>>logs.
>>
>>
>>
>>If a frog had pockets, he would carry a gun and not have to worry about
>>snakes, too.
>>
>>If...  If...  If...
>
>That's a bit unfair. If I try to explain something to you, I am forced to use
>such technique. It seems as if you didn't like the points I could demonstrate.
>
>
>>
>>
>>
>>
>>> Don't you see how far you must forget about your own experience to be able
>>>to argue against my crystal clear logic? Because, only for younger readers, you
>>>are by far _no_ beginner! You know exactly what went wrong with the whole event.
>>
>>
>>
>>
>>yes I do.  Absolutely nothing went wrong except that nobody likes the outcome.
>>Nobody complained (yourself included) about the 1996 event.  Same rules.  Same
>>secrecy.  Same lack of logs.  Same everything.  Oh yes.  The _result_ was
>>different so it was ok.
>>
>>But in 1997, _what_ was different?  :)
>>
>>only _one_ thing.
>
>Your memory is weak, but I can help you. IBM was sponsor and player!

IBM was sponsor _and_ player in 1996 too.  Or did you forget that?  Yet
nobody complained in 1996...


>
>
>
>>
>>>I do not talk about the question of who did what. But about the conditions of
>>>the event. And you know exactly that in such a case of
>>>not-to-prove-interdependencies you must hold up by force the friendly climate,
>>>otherwise you'd end in self-betrayal. This stands above the contracts, and you
>>>know that!
>>
>>
>>_nothing_ stands "above the contract".  You sign.  You are "in."
>
>(Sigh.) Climate. Ethics. Your choice!
>
>
>
>>
>>
>>
>>>
>>>
>>>>
>>>>
>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>> we would at least be able to
>>>>>>>understand why our logic is shaky when we conclude that if some PC progs could
>>>>>>>find the solution that then it might be possible for DB 2 as well. You know, we
>>>>>>>must not take Kasparov as our scientifical God but we should take him as the
>>>>>>>honest reporter from the chess angle of the problem. In other words. Even if you
>>>>>>>were right, that his "claim" could be refutated, he could still be right.
>>>>>>>Look, if that would come out in 40 years when you and me are dancing in
>>>>>>>paradise, it would be a pity if the security patrol would catch you for being
>>>>>>>guilty of supporting the confusion to the disadvantage of Kasparov, just because
>>>>>>>you are a friend of many of the DB 2 team.
>>>>>>
>>>>>>
>>>>>>I would defend deep blue whether I was _friends_ with the team or not.  It
>>>>>>is their _reputations_ that cause me to defend them, not the fact that I have
>>>>>>been on a first-name basis with them for 20+ years.
>>>>>
>>>>>Yes, but it's five years ago now that I told you that they probably violated all
>>>>>ethics of science, no matter if they were working for IBM or not. The details
>>>>>after game two speak against them. They should have known better. Alas, when Hsu
>>>>>Feng tried to get in touch with Kasparov we saw what his reputation was worth.
>>>>>He left the field for good. He will never more be accepted from a top
>>>>>chessplayer. What did Kasparov say? "You must qualify yourself, then come back."
>>>>
>>>>I wonder why Kasparov would not say "yes, I will play your machine for the
>>>>easy opportunity to pick up a million dollars."???
>>>
>>>Easy one, Hsu misused his confidence once. He wouldn't let himself be fooled
>>>twice. For the reasons I gave you. He did it "for" computerchess, not for the
>>>money, didn't you know that? Are you familiar with the psyche of a chess genius?
>>>He's a chessplayer in the first instance and a bad politician. And as a
>>>chessplayer he always liked and admired computerchess. That he didn't say what
>>>you suggested _proves_ that he's primarily not after the money. He already had
>>>enough money.
>>
>>If I thought I could win, I would play for $1,000,000 against _you_ and whatever
>>machine you had.  Regardless of whether I respected or despised you.  It would
>>be about the money.  Unless I was somehow "afraid"...
>
>But you can see that you can't buy a Kasparov! He will do many things if (!) it
>is ok with his chess interest. Here he's like Bobby who prefered to leave the
>tournament for good if something fishy against the players cold be foreseen.


Kasparov was bought and paid for _twice_.  In 1996 and 1997.  that happens
all the time, in pro sports for example.  Nothing wrong with using your talents
to cause someone to hire you for whatever reason...



>
>>
>>
>>
>>
>>>
>>>
>>>>
>>>>I can think of several reasons why he wouldn't.  And the main one _did_ have
>>>>something to do with Hsu's reputation.  Because Hsu told him he would have
>>>>a machine another 30X faster.  I don't think Kasparov wanted _any_ part of
>>>>that...
>>>
>>>And I say that inspite such a monster chess machine Hsu had exactly nothing in
>>>his hands _without_ Kaspy. Do you believe that Hsu had left the field if he had
>>>something out of himself? You bet.
>>
>>Hsu needed Kasparov to make the next potential match interesting.  Without
>>Kasparov, there would be no interest.  No interest means no financial backing.
>>
>>No mystery there.  Why do you think IBM got into this?  For the science?  Or
>>for the potential public relations bonanza?
>
>The point is this. You accept this logically, but then you must agree that
>Kasparov is the more important figure of the whole CC ballyhoo and not the
>machine! But you argue as if Kasparov had to grovel before Hsu et al. and to be
>thankful. At least here we've found agreement.

You have it backward.  Kasparov was _the_ best player in the world, and that
was what the fredkin prize was all about...  He held _every_ card in the deck,
and then he complains because he dealt himself a losing hand.  That was hardly
IBM's fault...




>
>
>>
>>
>>
>>
>>
>>>
>>>>
>>>>
>>>>
>>>>
>>>>>What does that mean? Oh, nothing but the _fact_ that only with the friendliness
>>>>>of Kasparov your friends got the chance to play. And then they violated the
>>>>>silent mutual acceptance that "normally" a show like that would never happen,
>>>>>simply because it's unfair to the bones.
>>>>
>>>>You don't think the chance to win $1,000,000.00 had _anything_ to do with
>>>>Kasparov's decision?  It had _nothing_ to do with friendship.  It had
>>>>_everything_ to do with $$$...
>>>>
>>>>Otherwise don't you think Friedel would have set up a match between Kasparov
>>>>and Fritz?  Eh?
>>>
>>>(Laughing.) But Friedel did the gig with Kramnik. I already criticised Kramnik
>>>for not letting Eduard play! But he wouldn't listen... :))
>>>The logic of Fred is that for FRITZ someone stronger than Kasparov had to be
>>>found. (Laughing.)
>>
>>I don't think he has found that someone yet.  Perhaps in 10 years...
>
>The irony is that Kramnik already was better than Kasparov, but during all the
>FRITZ preparation he became weaker and weaker. Just joking. But he has beaten
>Kasparov convincingly. 2:0!
>
>Please let's not debate without deeper thoughts. Give me something to bite! I
>thought that you were defending the DB2 team and IBM. :)
>
>Rolf Tueschen

I'm not defending IBM whatsoever.  I am defending the reputations of the people
involved in "the project"...

IBM has nothing to do with any of this, other than they supplied a _lot_ of
money to lure Kasparov into the match, and a lot of money to pay the group to
do the development, and a lot of money to pay for the hardware, and a lot of
money to do the pre-match publicity.  In return they got more advertising
and publicity than they could have bought by spending 100x that amount of
money.  Sound business, IMHO.

>
>
>>
>>>
>>>>
>>>>
>>>>
>>>>>
>>>>>You know, Bob, it's kind of strange, that you had to come forward with the
>>>>>"contracts". Of course you are right with the contracts. But at that moment you
>>>>>lost the track of the silent mutual acceptance. Their is a logic beyond
>>>>>contracts. In special with contracts for simple show events!
>>>>
>>>>I was taught _long_ ago.  "if it is important, get it in writing.  If it is
>>>>not important, then a wink and a nod is perfectly acceptable."  Kasparov signed
>>>>a contract after negotiating the details of that contract.  If you sign a
>>>>contract to buy a Rolls Royce, and after you get it, you decide "this thing
>>>>isn't worth this much money" it is _too late_.  You negotiate _before_ signing,
>>>>_not_ after...
>>>
>>>Excuse me for insisting, but you're really a bit rigid. :)
>>>We're talking about a little show event, not concrete economics or rocket
>>>science. The contracts were mainly about the chess clock, the funny ambience of
>>>the living room atmosphere and so on. The rest _was_ confidence among friends...
>>
>>Where did you get that?  The contract stipulated when and where each game would
>>be played, the time control, the number of rest days and where they would be
>>inserted, who could go to the bathroom when, you-name-it...
>>
>>Don't blame a lack of contract negotiation by Kasparov as a trick by IBM.
>>He _did_ play the previous year, _same_ contract rules...  But there was
>>a difference of course... the outcome.
>>
>>
>>
>>>Therefore you irony always was most unwanted. But what could Kasparov do when
>>>suddenly these chess beginners went completely gaga and thought that there baby
>>>could be stronger than Kasparov this time - in special after the famous first
>>>game? Not to speak about the marketing elites of IBM who didn't even know on how
>>>many squares they actually played in chess. (Just assumptions.)
>>>
>>>
>>>>
>>>>
>>>>
>>>>
>>>>>
>>>>>I think that this is the best possible description of what has happened. Without
>>>>>insults. The facts alone speak their verdict. Against one certain side.
>>>>>They changed a show act into a World Championship and then denied to pass the
>>>>>doping control. The End.
>>>>>
>>>>>
>>>>>Rolf Tueschen
>>>>>
>>>>
>>>>That is just your characterization.  Not the general opinion.
>>>
>>>Thanks for the compliment, Bob.
>>>
>>>Rolf Tueschen
>>>
>>>
>>>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Friendship is one thing but Science
>>>>>>>and Ethics is something more important. And I tell you, the truth will come out.
>>>>>>>Now or in future. And you won't have a good excuse with the statement that
>>>>>>>Kasparov made a scientifically wrong thesis. We all know that he's not a
>>>>>>>scientist. But he's one of the best chessplayers we ever had.
>>>>>>>
>>>>>>>Rolf Tueschen
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>And one of the biggest sore losers as well...
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>The deconstruction of DEEP BLUE 2 right after the event, in special with the
>>>>>>>>>knowledge of the prior attitude of the DB team, which was one of secrecy (not a
>>>>>>>>>single game score existing!), speaks against the validity of DB2 output.
>>>>>>>>>
>>>>>>>>>THe deliberate deconstruction invalidates DB2 results. Just compare it with the
>>>>>>>>>refusal of passing the doping test directly _after_ the race.
>>>>>>>>>
>>>>>>>>>It's so basic!
>>>>>>>>>
>>>>>>>>>Rolf Tueschen



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.