Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: LCT II Fin4, Deep Thought, and Deep Blue (was Re: LCT II results...)

Author: Robert Hyatt

Date: 19:51:59 01/07/98

Go up one level in this thread


On January 07, 1998 at 15:58:46, Don Dailey wrote:

>Hi Bob,
>
>A lot of your response I can pass off as a difference of opinion which
>is a good thing if learning takes place.   I just want to address a
>few issues:
>
>I mentioned something about Deep Blue not doing repetitions in the
>hardware part of the search.   You assumed I said quies part of the
>search.   My program also limits the rep tests, but not on the last
>4 ply of the MAIN search.   I think you just misread my section,
>you were probably just too eager to respond.

Actually I just assumed you wrote what I responded to.  While I could
be wrong, I thought that in a conversation with Hsu at Cape May, he
said they didn't catch repetitions once they started the quiescence
search, since they did follow some checking moves there.  I could be
remembering this incorrectly, but in the case of deep blue, it is a moot
point since DB never had this problem...


>
>The cute anecdote about Deep Blue and a private match would not
>impress any scientist.   I don't mind hearing these kind of
>stories when I'm bullshitting around (I do it too.)   But
>presenting this as some kind of refutation is ridiculous and
>I cannot let it pass.   First of all 10 games is way to small
>do draw a  solid conclusion from.  Some piece of hardware
>doing 100 thousand nodes still sounds like a mismatch to me
>(of course depending on the what the micros were running on
>which you didn't specify) and what time control, and who were
>the witnesses etc.

Here's my "evidence" since you'd prefer more scientific data:

1.  they have won over 9 of every 10 games against computers since they
started.  I gave a year by year summary for every machine-type event
they
played in.

2.  they've dominated computer chess events like no one else ever has.

3.  they have had outstanding match results against many GM players,
including robert byrne, benjamin, and at least a dozen others.  Some
of the matches were public exhibitions, some where not.  But the games
were
not "blitz"...

I agree that DB at 100K is not the same as a micro at 100K, because DB
gets their huge eval for free, and don't suffer the huge slowdown they
would on a micro, if they implemented their eval in software.  But the
point of Hsu's experiment was simply to answer the question, "if we
equalize the speed (as best we can) how well will we do against the
best of the commercial micros?"  I assume he was trying to answer this
because so many postulate the grossly inaccurate opinion that DB's eval
is "trivial" and all they have is "speed".  If nothing else, the 10-0
match shows they aren't just "fast"...  they have some hellacious smarts
as well...



>
>Tell me your stories because they are fun to hear, but please
>don't expect me to assign a great deal of weight to the conclusions
>you draw from them.

there you have to suit yourself.  I've learned to trust those that have
done this for a long time.  Personally, I'm not prone to bullshitting
anyone, and never have been.  The commercial programmers (in particular)
have made some ludicrous statements about how they compare to DB.  I
simply
refuse to let those stand unchallenged, when I *know* they are
incorrect.


>
>Next time I will want thorough documentation describing who
>did the experiment, the exact conditions of the experiment,
>who the witnesses were and so on.   If this already exists
>or was written up somewhere I will take a look and then I
>have a basis for changing my mind.  But I already know in
>advance that the 10 game sample is not enough.
>
>You also made it sound like a micro has no chance against
>a GM.   When is the last time you got out of the house?
>It would be good for you to get some air.

Read carefully:

  "GM players are *far* superior to *any* micro-based program, when the
   game is played at a tournament time control."

That's all I've said.  At tournament time controls, playing a *match*, I
stand by the "micros have no chance to win".  They might win the
occasional
game, but only a game.  And this is not including the super-GM players.
There
the micros are lucky to even win a game.  Who's beaten Kasparov in a
tournament
game?  Who's beaten Karpov?  Kramnik?  Adams?  Short?  Anand?  Topolov?
Etc.

The wins against *those* players are almost non-existant.  Yes, if you
go
to Aegon results, you can find a win here and there.  But don't forget
to
also tally all the losses...


>
>You mentioned in a later post that parallel programs are
>not that slow.  Now this is something I know about and
>you do too.  But you admitted a 3/4 slowdown for 16
>processors.  This is all I need to make my point that
>NPS for a parallel machine is not equivalent.  It should
>be noted that these are YOUR numbers (mine are similar)
>but they are not DEEP BLUE's numbers.  They do much
>worse than this.

No...  I admitted a 1/4 slowdown...

Our 16 processor program ran 12X faster than our 1 processor program.
Note that this is on an architecture unlike those you have been using,
and that my "latency" is "0" which certainly helps.  3/4 efficiency is
not bad at all.  Hsu claims 33% efficiency, which sounds right because
he has more latency than I do, eith a two-level parallel search,
partially
divided on the SP processors, and then sub-divided further among the
chess processors connected to each SP cpu...

But 33% efficiency is hellaciously good, because 1/3 of their machine
is one hell of a lot of searching.  If they were only 10% efficient,
they would still blow us all away in raw speed...  that was my point.

So if they are 1/3 efficient, and are running with 256 processors, they
act like they are running with only 85 processors.  Of course 85
processors
at 2.4M nodes per second per processor is pretty good...

>
>You claimed micros make more compromises than deep blue.
>Please don't insult my intelligence.  Don't assume I
>know nothing about the issues they deal with.  I'll
>give you and the readers an example of the type of
>compromise they must take.  The hash table implementation
>is different in the hardware portion (last few plies)
>of the search.   They use small local tables.  This
>is not what you would like to do (global tables are best)
>but it's a good compromise for them, otherwise they
>would have to give up too much speed.   I talked to
>them at the Deep Blue match and they were not even
>using it at the time, I think there was a bug or
>something, I don't really know why.   This is a typical
>tradeoff you're forced to make with hardware.  The
>end result is good, that's why you do it in the first
>place.  But saying they didn't have to compromise anything
>is simply inacurate.


they didn't *have* to do this.  This was a simple choice they chose
to make.  IE I run on a Cray with 32 processors with fully shared
memory.  Nothing prevented them from doing so, had they wanted to
do so.  They did just like we do, they weigh the advantage of sharing
memory among all processors vs the overhead of synchronizing the memory
ports to accomplish this, and then decide "lets take speed over
efficiency
because we believe that we can get enough more speed to offset the
efficiency
issue"...

I didn't say they don't make compromises... I said they don't make
compromises *just* because they are doing hardware.  They make
compromises for efficiency or simplicity, just like we do.  But I
can assure you that if they wanted a global hash table, it could be
done... Cray did it.  So could they.  Implying that they have to re-fab
for every change to the code is also wrong.  Eval is already "soft".  I
doubt they change their q-search any more than I have in over a year.
So
all their work can go into the SP search code, just like I've been doing
on the basic search in Crafty.

Also, the design is not nearly so complex a process as you might
suspect.
I've done some of that, and silicon compilers make the task a lot less
complicated than thinking about "gates" and "paths"...  And there are
tools to help modify a design to eliminate propogation delays and the
like...  very similar to programs I used on the Cray to detect memory
delays and instruction issue delays...


>
>The final point I want to make is that I'm not in the
>"deep blue sucks" camp.   I got the idea you percieved
>my post as an attack on  YOU.   I don't know why, I
>was talking about Deep Blue, didn't even mention your
>name, and even then I wasn't attacking it.

sorry if I came off that way.  I don't have anything to do with the
DB project, so no way you could say something about them, and attack
me.  I simply try to correct mis-statements that pop up every six months
or so.  What they are doing is not that different than what we are
doing,
except we have different trade-offs we have to make.  They ignore the
eval speed issues since they can do all this in parallel in the
hardware.
But they are fighting parallel search issues that the sequential
programs
arent, so they just shift concerns from one issue to another.  But there
is *nothing* to indicate that any of the following are true:

1.  that their search or evaluation is inferior to any program currently
running.  In fact, the contrary is true.

2.  that because they are designing hardware, they are losing
flexibility.
I'd love to have my q-search in hardware.  I'd never change it anyway,
and
would like to go that fast.

3.  that they have compromised many things to go to hardware.  The
inverse
is true, they actually gain by doing this, rather than losing.  I'm
working
on attack detection in crafty and it is pretty costly to figure out what
is bearing on a king, particular thru other pieces.  They did this for
a total cost of "0".  With them, it's a matter of dreaming "what do I
want
to do" because the hardware cost is zilch, rather than our dreaming
"what
*can* I do that is not going to slow me down so much it offsets the
gain?"



>
>It was like your post was designed to make it seem like
>I was attacking deep blue (and you.)

Sorry...  was *not* intended to be so...


>
>- Don
>
>
>
>On January 06, 1998 at 20:10:07, Robert Hyatt wrote:
>
>>On January 06, 1998 at 17:30:40, Don Dailey wrote:
>>
>>
>>>
>>>Hi guys,
>>>
>>>Here is my take on Deep Blue and it's algorithms.  First of all their
>>>approach is based on lots of hardware which gives them a HUGE problem.
>>>If something is wrong with our software we quickly fix it.  If something
>>>is wrong with their hardware they have a huge problem that will take
>>>months to get to the next version.   So there is a lot of flexibility
>>>we take for granted that they do not get.  What they have done is very
>>>impressive indeed and took a great deal of engineering talent.
>>
>>don't take this "too" far.  Much of what they do was microcode, and I
>>doubt
>>this has changed, so that many things can be changed.  Also, they use
>>the
>>chess hardware out near the tips.  The main search is done on the IBM
>>SP.
>>IE they look a lot like Crafty, the hardware handles the last N plies in
>>a simplistic manner, plus the capture search.  The SP (software) handles
>>the first M plies, does the singular extension stuff and everything else
>>they do.
>>
>>They wouldn't need to modify the search in the hardware very often.  The
>>evaluation is already programmable, so that can be changed easily and
>>often
>>without modifying the hardware at all...
>>
>>
>>
>>>
>>>As far as the classic question about how would they do against the
>>>best micro's on equal hardware?   First of all it's not easy to define
>>>what equal hardware is at all.   But I'll take a stab and give you
>>>my sense of the issues involved.
>>>
>>>Let's use REBEL as representative of the best software available.
>>>If you scaled Rebel up to do the same Nodes per second as Deep Blue
>>>there would be no contest, Rebel would be a HUGE favorite.
>>
>>I *totally* disagree.  I'd peg DB's evaluation at being approximately
>>100X more complex than Rebel's based on Ed's current NPS rate.  Don't
>>forget the match that caused so much discussion where a single DB
>>processor
>>running at 100K nodes per second really smacked Rebel and Genius 10-0 in
>>Hsu's lab.  So you are *way* off base here.  *way* off...
>>
>>
>>
>>>
>>>But this is hardly a fair comparison, Rebel is a SERIAL program and
>>>is clearly more efficient than a parallel program which tends to look
>>>at many extra nodes to do the same amount of effective processing.
>>>
>>>So let's "pretend" we can run the pure Deep Blue algorithm in SERIAL
>>>mode and match up both Rebel and Deep Blue, let's say 2 million nodes
>>>per second (and equal hash tables.)
>>>
>>>The winner?   REBEL wins again!  But we are still being quite unfair.
>>>Deep Blue is forced to accept compromises and inflexibilities that
>>>REBEL does not have to deal with.  It's quite certain that many design
>>>choices were optimized for the exact approach each side was using.
>>>From Deep Blues point of view, the stuff in Rebel would be wrong to
>>>attempt to implement in Deep Blue.
>>
>>So far as I know, there are *no* compromises in DB.  They have done
>>*everything* they wanted...  evaluation, attack detection, threat
>>detection, I mean *everything*...
>>
>>Again, you are taking your knowledge of Rebel, but comparing it to
>>almost no knowledge about DB.  That gadget is *far* more sophisticated
>>than anything else currently playing chess.  Not just faster, but
>>smarter
>>as well.
>>
>>
>>>
>>>An example of this will suffice.  Until recently Deep Blue could not
>>>even pick up repetition in the hardware portion of the search.  No micro
>>>program would dare leave this out, it's a bad idea.  But at the time
>>>choosing to leave it out seemed right for Deep Blue because it added
>>>too much complexity to the chips that did the end node searching.
>>>When we played them in Hong Kong they were quite afraid we might get
>>>a draw (we did not) because there were long checking lines for us.
>>>They were noticably disturbed by the possibility.
>>
>>
>>Guess again.  Crafty doesn't catch repetitions in the q-search, because
>>they are *impossible* in my q-search, which only includes captures and
>>promotions.  Ditto for Ferret.  We're both doing reasonably well.  They
>>have handled repetitions correctly since Deep Blue started playing.  The
>>older Deep Thought and "deep blue prototype" had that bug, if you call
>>it
>>that... but it wasn't a serious issue.
>>
>>>
>>>Well since then they have corrected this problem but there was no
>>>easy fix.  It took a complete re-engineering of the chip and probably
>>>at least a YEAR or more to go through the whole cycle.
>>>
>>>The real bottom line here is that it is almost silly to compare the
>>>two programs except on absolute strength.   Deep Blue could probably
>>>not hold up MOST of top micro's if you tried to equalize everything
>>>in this manner but it's no reflection on the Deep Blue team.   In
>>>every way (except raw speed) the Deep Blue team is handicapped so you
>>>can not expect them to compete with the highly tuned micro programs.
>>
>>Don, your lack of hardware design experience shows here, no insult
>>intended.  They can do *anything* they want, and with "silicon
>>compilers"
>>it is trivial to do for them.  Hardware design is now more like
>>programming
>>than designing.  But there are fewer compromises in DB than in the micro
>>programs.
>>
>>IE I'd *love* to design such a chip, because my rotated bitmaps would be
>>perfect for that type of hardware, because I could do the rotation in 0
>>cycles.  In fact, a "crafty on a chip" would not be difficult to do, if
>>there was funding to pay the bill.  But your "DB is full of compromises"
>>is simply off-base.  You ought to poke Hsu over the phone or via email
>>to
>>get a better feel for what they have done.  It's most impressive...  and
>>not just because it is fast...  They have it *all*...
>>
>>
>>
>>>
>>>Would you compare a world class human sprinter to a cheetah and say
>>>how fast would the Cheetah be if it were only human?
>>>
>>>So does Deep Blue suck?   In rating points per node searched, YES.
>>>In absolute strength of course NOT.  It's unclear (to me) if they
>>>are much better than the very best micro's but I'm pretty sure it
>>>would win a long match against any of them (this year anyway.)
>>>
>>>Deep Blue's performance seems to be about as good as the top micro's
>>>based on the few tournaments it's played in and the close (but very
>>>short) match against Kasparov is a good indication that it's quite
>>>strong.
>>
>>This I don't follow.  What micro has beaten a GM in 40/2?  In a match
>>of 40/2?  What micro has beaten as many GM's as DB in anything
>>(excepting
>>blitz, where most micros do ok at times)...
>>
>>
>>>
>>>Sorry Bruce, I know you didn't want to hear about this!   I carefully
>>>avoided singing their praises or saying they sucked!
>>>
>>>- Don



This page took 0.02 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.