Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: LCT II Fin4, Deep Thought, and Deep Blue (was Re: LCT II results...)

Author: Don Dailey

Date: 00:23:10 01/08/98

Go up one level in this thread


>>I mentioned something about Deep Blue not doing repetitions in the
>>hardware part of the search.   You assumed I said quies part of the
>>search.   My program also limits the rep tests, but not on the last
>>4 ply of the MAIN search.   I think you just misread my section,
>>you were probably just too eager to respond.
>
>Actually I just assumed you wrote what I responded to.  While I could
>be wrong, I thought that in a conversation with Hsu at Cape May, he
>said they didn't catch repetitions once they started the quiescence
>search, since they did follow some checking moves there.  I could be
>remembering this incorrectly, but in the case of deep blue, it is a moot
>point since DB never had this problem...

I'm not sure if he told me this in Hong Kong or Cape May.  But at
any rate it is not a moot point, because I am using it as an example
of how difficult there development cycle must be.  I knew when I
posted it that the problem had been solved and I even said this.

>Here's my "evidence" since you'd prefer more scientific data:
>
>1.  they have won over 9 of every 10 games against computers since they
>started.  I gave a year by year summary for every machine-type event
>they
>played in.

9 out of 10 is a good score but not a great score.  You should
realize this.  I'll bet Cray Blitz had similar numbers but I
don't think it was hundreds of rating points stronger than the
very best.

The problem is that most of the field is "easy pickings" for them
AND for the other TOP programs too.  If most of the games are
this way then their score against QUALITY opposition is more like
2 out of 3  or something not nearly as impressive.  But if I'm
generous and say Deep Blue wins 4 out of 5 against the top 3
programs, there is a problem that you cannot resolve.  They are
doing roughly 500 times more nodes than these programs AND there
evaluation BLOWS away every one elses AND there are no compromises
they take, the parallelism doesn't even slow them down much and
yet they lose once every 4 or 5 games against the good ones?  I
have tested my program against itself and against Zarkov and the
score quickly get's over 95% with 2 to 3 doublings of speed.
Against Zarkov it needs an extra doubling because my program needs
the first doubling to break even (don't worry, I'm about 2X faster)

Oh what the hell, let's say it wins 9.5 out of 10 and assume every
opponent is as good as Rebel or King or whoever.  The numbers
still don't add up the way you think they do despite the fact that
the micro's have to take more compromises than DB according to you.

In the last two computer vs computer events Cilkchess played in
we have just about the same numbers.  (10 out of 11 and 9.5 out of
11.)   We played 22 games and lost only 1 game and only had 3 draws.
I perf rated these games and was expecting huge numbers but this
didn't happen.  They were good but not several hundred over every
one else.  There was a rather large gap between smallest and largest.

>2.  they've dominated computer chess events like no one else ever has.

I think the Northwestern program may have had similar results but
maybe not quite as good.  Your statement would make it seem their
domination was nothing at all compared to Deep Blues.

>
>3.  they have had outstanding match results against many GM players,
>including robert byrne, benjamin, and at least a dozen others.  Some
>of the matches were public exhibitions, some where not.  But the games
>were
>not "blitz"...

Hey, I'm never going to argue against this.  I know they have had
outstanding results.  STOP defending them, I'm not attacking them.


>I agree that DB at 100K is not the same as a micro at 100K, because DB
>gets their huge eval for free, and don't suffer the huge slowdown they
>would on a micro, if they implemented their eval in software.  But the
>point of Hsu's experiment was simply to answer the question, "if we
>equalize the speed (as best we can) how well will we do against the
>best of the commercial micros?"  I assume he was trying to answer this
>because so many postulate the grossly inaccurate opinion that DB's eval
>is "trivial" and all they have is "speed".  If nothing else, the 10-0
>match shows they aren't just "fast"...  they have some hellacious smarts
>as well...

And they are 500 times faster.  Like I said the numbers are not adding
up quite right.


>>Tell me your stories because they are fun to hear, but please
>>don't expect me to assign a great deal of weight to the conclusions
>>you draw from them.
>
>there you have to suit yourself.  I've learned to trust those that have
>done this for a long time.  Personally, I'm not prone to bullshitting
>anyone, and never have been.  The commercial programmers (in particular)
>have made some ludicrous statements about how they compare to DB.  I
>simply
>refuse to let those stand unchallenged, when I *know* they are
>incorrect.

This comparison is a form of flattery.


>>You also made it sound like a micro has no chance against
>>a GM.   When is the last time you got out of the house?
>>It would be good for you to get some air.
>
>Read carefully:
>
>  "GM players are *far* superior to *any* micro-based program, when the
>   game is played at a tournament time control."

Ok, I'll concede this point and it's an accurate statement.  Very soon
you will have to delete the *far* and shortly after this the whole
statement.  But you know this.   (Please no argument on when that will
be!)


>>You mentioned in a later post that parallel programs are
>>not that slow.  Now this is something I know about and
>>you do too.  But you admitted a 3/4 slowdown for 16
>>processors.  This is all I need to make my point that
>>NPS for a parallel machine is not equivalent.  It should
>>be noted that these are YOUR numbers (mine are similar)
>>but they are not DEEP BLUE's numbers.  They do much
>>worse than this.
>
>No...  I admitted a 1/4 slowdown...

That was a misprint Bob, I meant 3/4 the iteration speed.  I get the
same numbers roughly but it varies from position to position.  Sometimes
we get superlinear speedups especially with small number of processors
because a fast cutoff occurs but not the usual case.  I'm sure you
experience the same things.   I was quite surprised how easy it is
to write a parallel program.  I spent a LOT of time debugging stuff
before the Dutch tournament but none on the parallel stuff.

I'll try to get some more accurate numbers for you when I get a chance
to compare.  I'll several positions with different natures and different
numbers of processors.

>But 33% efficiency is hellaciously good, because 1/3 of their machine
>is one hell of a lot of searching.  If they were only 10% efficient,
>they would still blow us all away in raw speed...  that was my point.

I'm surprised it's that high with 2 levels and that many processors.

>they didn't *have* to do this.  This was a simple choice they chose
>to make.  IE I run on a Cray with 32 processors with fully shared
>memory.  Nothing prevented them from doing so, had they wanted to
>do so.  They did just like we do, they weigh the advantage of sharing
>memory among all processors vs the overhead of synchronizing the memory
>ports to accomplish this, and then decide "lets take speed over
>efficiency
>because we believe that we can get enough more speed to offset the
>efficiency
>issue"...

Of course they made choices just like we do.  Does this make my
argument invalid that Deep Blue needs more nodes to be equivalent?

I have never argued that they SHOULDN'T do this.  I'm arguing that
if you build hardware to compete with a Micro, make sure you look
at more nodes per second than they do for reasons like this one.


Bob, we could go on and on and on ...  but let's not.  I've stated
my case by now and will quit with this.   Don't think I'm attacking
Deep Blue, I just felt like you were out of balance with your
praises of Deep Blue.  Pehaps it struck a nerve with me because I
have felt this kind of hype is really bad for computer chess in
general.  It is similar to the checkers thing with Dr. Samuels
where for decades everyone believed computers had solved the game.



- Don




This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.