Author: Robert Hyatt
Date: 19:51:59 01/07/98
Go up one level in this thread
On January 07, 1998 at 15:58:46, Don Dailey wrote: >Hi Bob, > >A lot of your response I can pass off as a difference of opinion which >is a good thing if learning takes place. I just want to address a >few issues: > >I mentioned something about Deep Blue not doing repetitions in the >hardware part of the search. You assumed I said quies part of the >search. My program also limits the rep tests, but not on the last >4 ply of the MAIN search. I think you just misread my section, >you were probably just too eager to respond. Actually I just assumed you wrote what I responded to. While I could be wrong, I thought that in a conversation with Hsu at Cape May, he said they didn't catch repetitions once they started the quiescence search, since they did follow some checking moves there. I could be remembering this incorrectly, but in the case of deep blue, it is a moot point since DB never had this problem... > >The cute anecdote about Deep Blue and a private match would not >impress any scientist. I don't mind hearing these kind of >stories when I'm bullshitting around (I do it too.) But >presenting this as some kind of refutation is ridiculous and >I cannot let it pass. First of all 10 games is way to small >do draw a solid conclusion from. Some piece of hardware >doing 100 thousand nodes still sounds like a mismatch to me >(of course depending on the what the micros were running on >which you didn't specify) and what time control, and who were >the witnesses etc. Here's my "evidence" since you'd prefer more scientific data: 1. they have won over 9 of every 10 games against computers since they started. I gave a year by year summary for every machine-type event they played in. 2. they've dominated computer chess events like no one else ever has. 3. they have had outstanding match results against many GM players, including robert byrne, benjamin, and at least a dozen others. Some of the matches were public exhibitions, some where not. But the games were not "blitz"... I agree that DB at 100K is not the same as a micro at 100K, because DB gets their huge eval for free, and don't suffer the huge slowdown they would on a micro, if they implemented their eval in software. But the point of Hsu's experiment was simply to answer the question, "if we equalize the speed (as best we can) how well will we do against the best of the commercial micros?" I assume he was trying to answer this because so many postulate the grossly inaccurate opinion that DB's eval is "trivial" and all they have is "speed". If nothing else, the 10-0 match shows they aren't just "fast"... they have some hellacious smarts as well... > >Tell me your stories because they are fun to hear, but please >don't expect me to assign a great deal of weight to the conclusions >you draw from them. there you have to suit yourself. I've learned to trust those that have done this for a long time. Personally, I'm not prone to bullshitting anyone, and never have been. The commercial programmers (in particular) have made some ludicrous statements about how they compare to DB. I simply refuse to let those stand unchallenged, when I *know* they are incorrect. > >Next time I will want thorough documentation describing who >did the experiment, the exact conditions of the experiment, >who the witnesses were and so on. If this already exists >or was written up somewhere I will take a look and then I >have a basis for changing my mind. But I already know in >advance that the 10 game sample is not enough. > >You also made it sound like a micro has no chance against >a GM. When is the last time you got out of the house? >It would be good for you to get some air. Read carefully: "GM players are *far* superior to *any* micro-based program, when the game is played at a tournament time control." That's all I've said. At tournament time controls, playing a *match*, I stand by the "micros have no chance to win". They might win the occasional game, but only a game. And this is not including the super-GM players. There the micros are lucky to even win a game. Who's beaten Kasparov in a tournament game? Who's beaten Karpov? Kramnik? Adams? Short? Anand? Topolov? Etc. The wins against *those* players are almost non-existant. Yes, if you go to Aegon results, you can find a win here and there. But don't forget to also tally all the losses... > >You mentioned in a later post that parallel programs are >not that slow. Now this is something I know about and >you do too. But you admitted a 3/4 slowdown for 16 >processors. This is all I need to make my point that >NPS for a parallel machine is not equivalent. It should >be noted that these are YOUR numbers (mine are similar) >but they are not DEEP BLUE's numbers. They do much >worse than this. No... I admitted a 1/4 slowdown... Our 16 processor program ran 12X faster than our 1 processor program. Note that this is on an architecture unlike those you have been using, and that my "latency" is "0" which certainly helps. 3/4 efficiency is not bad at all. Hsu claims 33% efficiency, which sounds right because he has more latency than I do, eith a two-level parallel search, partially divided on the SP processors, and then sub-divided further among the chess processors connected to each SP cpu... But 33% efficiency is hellaciously good, because 1/3 of their machine is one hell of a lot of searching. If they were only 10% efficient, they would still blow us all away in raw speed... that was my point. So if they are 1/3 efficient, and are running with 256 processors, they act like they are running with only 85 processors. Of course 85 processors at 2.4M nodes per second per processor is pretty good... > >You claimed micros make more compromises than deep blue. >Please don't insult my intelligence. Don't assume I >know nothing about the issues they deal with. I'll >give you and the readers an example of the type of >compromise they must take. The hash table implementation >is different in the hardware portion (last few plies) >of the search. They use small local tables. This >is not what you would like to do (global tables are best) >but it's a good compromise for them, otherwise they >would have to give up too much speed. I talked to >them at the Deep Blue match and they were not even >using it at the time, I think there was a bug or >something, I don't really know why. This is a typical >tradeoff you're forced to make with hardware. The >end result is good, that's why you do it in the first >place. But saying they didn't have to compromise anything >is simply inacurate. they didn't *have* to do this. This was a simple choice they chose to make. IE I run on a Cray with 32 processors with fully shared memory. Nothing prevented them from doing so, had they wanted to do so. They did just like we do, they weigh the advantage of sharing memory among all processors vs the overhead of synchronizing the memory ports to accomplish this, and then decide "lets take speed over efficiency because we believe that we can get enough more speed to offset the efficiency issue"... I didn't say they don't make compromises... I said they don't make compromises *just* because they are doing hardware. They make compromises for efficiency or simplicity, just like we do. But I can assure you that if they wanted a global hash table, it could be done... Cray did it. So could they. Implying that they have to re-fab for every change to the code is also wrong. Eval is already "soft". I doubt they change their q-search any more than I have in over a year. So all their work can go into the SP search code, just like I've been doing on the basic search in Crafty. Also, the design is not nearly so complex a process as you might suspect. I've done some of that, and silicon compilers make the task a lot less complicated than thinking about "gates" and "paths"... And there are tools to help modify a design to eliminate propogation delays and the like... very similar to programs I used on the Cray to detect memory delays and instruction issue delays... > >The final point I want to make is that I'm not in the >"deep blue sucks" camp. I got the idea you percieved >my post as an attack on YOU. I don't know why, I >was talking about Deep Blue, didn't even mention your >name, and even then I wasn't attacking it. sorry if I came off that way. I don't have anything to do with the DB project, so no way you could say something about them, and attack me. I simply try to correct mis-statements that pop up every six months or so. What they are doing is not that different than what we are doing, except we have different trade-offs we have to make. They ignore the eval speed issues since they can do all this in parallel in the hardware. But they are fighting parallel search issues that the sequential programs arent, so they just shift concerns from one issue to another. But there is *nothing* to indicate that any of the following are true: 1. that their search or evaluation is inferior to any program currently running. In fact, the contrary is true. 2. that because they are designing hardware, they are losing flexibility. I'd love to have my q-search in hardware. I'd never change it anyway, and would like to go that fast. 3. that they have compromised many things to go to hardware. The inverse is true, they actually gain by doing this, rather than losing. I'm working on attack detection in crafty and it is pretty costly to figure out what is bearing on a king, particular thru other pieces. They did this for a total cost of "0". With them, it's a matter of dreaming "what do I want to do" because the hardware cost is zilch, rather than our dreaming "what *can* I do that is not going to slow me down so much it offsets the gain?" > >It was like your post was designed to make it seem like >I was attacking deep blue (and you.) Sorry... was *not* intended to be so... > >- Don > > > >On January 06, 1998 at 20:10:07, Robert Hyatt wrote: > >>On January 06, 1998 at 17:30:40, Don Dailey wrote: >> >> >>> >>>Hi guys, >>> >>>Here is my take on Deep Blue and it's algorithms. First of all their >>>approach is based on lots of hardware which gives them a HUGE problem. >>>If something is wrong with our software we quickly fix it. If something >>>is wrong with their hardware they have a huge problem that will take >>>months to get to the next version. So there is a lot of flexibility >>>we take for granted that they do not get. What they have done is very >>>impressive indeed and took a great deal of engineering talent. >> >>don't take this "too" far. Much of what they do was microcode, and I >>doubt >>this has changed, so that many things can be changed. Also, they use >>the >>chess hardware out near the tips. The main search is done on the IBM >>SP. >>IE they look a lot like Crafty, the hardware handles the last N plies in >>a simplistic manner, plus the capture search. The SP (software) handles >>the first M plies, does the singular extension stuff and everything else >>they do. >> >>They wouldn't need to modify the search in the hardware very often. The >>evaluation is already programmable, so that can be changed easily and >>often >>without modifying the hardware at all... >> >> >> >>> >>>As far as the classic question about how would they do against the >>>best micro's on equal hardware? First of all it's not easy to define >>>what equal hardware is at all. But I'll take a stab and give you >>>my sense of the issues involved. >>> >>>Let's use REBEL as representative of the best software available. >>>If you scaled Rebel up to do the same Nodes per second as Deep Blue >>>there would be no contest, Rebel would be a HUGE favorite. >> >>I *totally* disagree. I'd peg DB's evaluation at being approximately >>100X more complex than Rebel's based on Ed's current NPS rate. Don't >>forget the match that caused so much discussion where a single DB >>processor >>running at 100K nodes per second really smacked Rebel and Genius 10-0 in >>Hsu's lab. So you are *way* off base here. *way* off... >> >> >> >>> >>>But this is hardly a fair comparison, Rebel is a SERIAL program and >>>is clearly more efficient than a parallel program which tends to look >>>at many extra nodes to do the same amount of effective processing. >>> >>>So let's "pretend" we can run the pure Deep Blue algorithm in SERIAL >>>mode and match up both Rebel and Deep Blue, let's say 2 million nodes >>>per second (and equal hash tables.) >>> >>>The winner? REBEL wins again! But we are still being quite unfair. >>>Deep Blue is forced to accept compromises and inflexibilities that >>>REBEL does not have to deal with. It's quite certain that many design >>>choices were optimized for the exact approach each side was using. >>>From Deep Blues point of view, the stuff in Rebel would be wrong to >>>attempt to implement in Deep Blue. >> >>So far as I know, there are *no* compromises in DB. They have done >>*everything* they wanted... evaluation, attack detection, threat >>detection, I mean *everything*... >> >>Again, you are taking your knowledge of Rebel, but comparing it to >>almost no knowledge about DB. That gadget is *far* more sophisticated >>than anything else currently playing chess. Not just faster, but >>smarter >>as well. >> >> >>> >>>An example of this will suffice. Until recently Deep Blue could not >>>even pick up repetition in the hardware portion of the search. No micro >>>program would dare leave this out, it's a bad idea. But at the time >>>choosing to leave it out seemed right for Deep Blue because it added >>>too much complexity to the chips that did the end node searching. >>>When we played them in Hong Kong they were quite afraid we might get >>>a draw (we did not) because there were long checking lines for us. >>>They were noticably disturbed by the possibility. >> >> >>Guess again. Crafty doesn't catch repetitions in the q-search, because >>they are *impossible* in my q-search, which only includes captures and >>promotions. Ditto for Ferret. We're both doing reasonably well. They >>have handled repetitions correctly since Deep Blue started playing. The >>older Deep Thought and "deep blue prototype" had that bug, if you call >>it >>that... but it wasn't a serious issue. >> >>> >>>Well since then they have corrected this problem but there was no >>>easy fix. It took a complete re-engineering of the chip and probably >>>at least a YEAR or more to go through the whole cycle. >>> >>>The real bottom line here is that it is almost silly to compare the >>>two programs except on absolute strength. Deep Blue could probably >>>not hold up MOST of top micro's if you tried to equalize everything >>>in this manner but it's no reflection on the Deep Blue team. In >>>every way (except raw speed) the Deep Blue team is handicapped so you >>>can not expect them to compete with the highly tuned micro programs. >> >>Don, your lack of hardware design experience shows here, no insult >>intended. They can do *anything* they want, and with "silicon >>compilers" >>it is trivial to do for them. Hardware design is now more like >>programming >>than designing. But there are fewer compromises in DB than in the micro >>programs. >> >>IE I'd *love* to design such a chip, because my rotated bitmaps would be >>perfect for that type of hardware, because I could do the rotation in 0 >>cycles. In fact, a "crafty on a chip" would not be difficult to do, if >>there was funding to pay the bill. But your "DB is full of compromises" >>is simply off-base. You ought to poke Hsu over the phone or via email >>to >>get a better feel for what they have done. It's most impressive... and >>not just because it is fast... They have it *all*... >> >> >> >>> >>>Would you compare a world class human sprinter to a cheetah and say >>>how fast would the Cheetah be if it were only human? >>> >>>So does Deep Blue suck? In rating points per node searched, YES. >>>In absolute strength of course NOT. It's unclear (to me) if they >>>are much better than the very best micro's but I'm pretty sure it >>>would win a long match against any of them (this year anyway.) >>> >>>Deep Blue's performance seems to be about as good as the top micro's >>>based on the few tournaments it's played in and the close (but very >>>short) match against Kasparov is a good indication that it's quite >>>strong. >> >>This I don't follow. What micro has beaten a GM in 40/2? In a match >>of 40/2? What micro has beaten as many GM's as DB in anything >>(excepting >>blitz, where most micros do ok at times)... >> >> >>> >>>Sorry Bruce, I know you didn't want to hear about this! I carefully >>>avoided singing their praises or saying they sucked! >>> >>>- Don
This page took 0.02 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.