Author: Vincent Diepeveen
Date: 03:02:08 06/26/98
Go up one level in this thread
On June 26, 1998 at 00:44:08, Don Dailey wrote: >Hi Vincent, > >This post is not specifically for you, but your post touches on these >points. So here goes ... > >Ok, it's thought experiment time! I would like to address what I call >the "knowledge myth" concerning chess programs. I think a lot of us >think of knowledge as a "quantity" and that simply adding more and >more makes your program play better and better. I will argue that >this is far from clear. > >Suppose your chess program was magically given INFINITE time to >perform any STATIC evaluation function you wanted to implement, but >everything else remained the same. Would it now suddenly become >trivial to write a super grandmaster, Kasparov beating monster on >modest hardware? > >You could do all those wonderful things you always wanted to do with >no speed penalty, the sky is the limit. Some of you have probably >already figured out that you could use this time to do more searching >but let's say this is not allowed, no cheating. > >I can think of lots of imaginative things I might try but in the end I >am afraid I would get only a modest improvement over what I have now >and most of this might well be because my current evaluation is not >free. But my point is, that we do not have a serious problem with >QUANTITY of knowledge, although certainly more would help, especially >if we can get it for free. The real problem is QUALITY! Does anyone >know how to write this new evaluation function? Is it as simple as >just adding more and more rules and somehow cramming in more QUANTITY? >NO it's not that simple. Right now the really good programs are >having a "law of diminishing returns" crisis. You can double the >knowledge (whatever that really means I don't know) but don't expect >more than tiny hand full of rating points. If you want the rating >points, you must do BETTER knowledge, not more. It's quantity versus >quality here. > >Here are some examples to make my point clear. Suppose my program >thinks knights are worth 1/2 a pawn. I keep watching it play these >unsound exchanges of knight for a pawn and wonder what is wrong. Is >my program lacking knowledge or does it have BAD knowledge? With my >infinite evaluator can you tell me what its true value should be? I >don't think you can. > >One more example. Endgame databases are an example of PERFECT >knowledge and most databases contain thousands if not millions of >terms or parameters or whatever you choose to call them. And yet >despite this massive amount of knowledge, they do not contribute more >than a few rating points to a chess program. And this is with PERFECT >knowledge. Most of the strength they do contribute is concentrated >among 2 or 3 common endings. It's a whole lot of knowledge, but >rarely used. > >There will quickly come a point when the bad knowledge that ALL our >programs have will place a bound on its possible strength (given a >certain amount of hardware.) Learning lots of rare endings and how to >play them will help in tiny incremental ways but never make up for the >other problems that are more QUALITATIVE. > >Saying Deep Blue or any other program has a billion parameters in it's >evaluation function doesn't say much of anything about how good its >evaluation is. > >Bob makes a strong case that Deep Blue is not limited to simplistic >evaluation just because it is implemented in hardware and in fact may >contain more terms than any other program. This is great and I'm all >for it, wish I had it too. But it's really hard for me to draw the >conclusion that it MUST be better because of this. This is an issue >that must be considered separately I consider the term count a minor >point although not totally irrelevant. (Don't blast me Bob because I'm >not saying Deep Blue sucks either, I certainly appreciate the point >you are making with Vincent that hardware evaluation does not equal >STUPID program.) > > >SUMMARY: > >. QUALITY much more important than QUANTITY > >. Law of diminishing returns, once you have a GOOD evaluation > function, it's hard to make it much better. > >. Knowledge is too hard to QUANTIFY. >- Don Yes it's very true, if you add new knowledge be sure it's knowledge and not heuristic. A known error in chessprograms (where some commercial suffer from) is for example to give queens points for distance to king of the opponent. This is a HEURISTIC. So there are a *lot* of cases where this is not true. It's hard to discriminate between knowledge and heuristic, and even for a good implementation of knowledge usual there is a position where it is not true, then quantity comes in. Currently most commercial programs most knowledge that's in is really knowledge, and not heuristic, but at the moment that you have a huge quantity of knowledge other parts of evaluation will compensate few patterns that go wrong. Still there is something lacking: strategy and overall board insight. Strategy is easy to define: that's the plan of the game. In opening usual simple plan (development), in endgames sometimes hard to make a program clear. Overall board insight is harder to define. for example you are getting slowly mated. Now your insight will say: by doing nothing i will lose, so you will develop a plan for example to either break the centre or try to mate your opponents king at the other wing. That's of course also a plan, but even when lacking a plan it's simple a reaction on what your opponent does or what position is asking for. So to answer your question, yes quantity is important and quality too. Quantity of quality makes a program really stronger. Yet we have to do something about the strategy humans are using, and their common sense. > > > >On June 25, 1998 at 20:52:40, Vincent Diepeveen wrote: > >> >>On June 25, 1998 at 12:15:09, Robert Hyatt wrote: >> >>>On June 25, 1998 at 08:07:53, Roberto Waldteufel wrote: >>> >>>> >>>>On June 25, 1998 at 06:35:40, Vincent Diepeveen wrote: >>>> >>>>> >>>>>On June 25, 1998 at 04:54:02, Roberto Waldteufel wrote: >>>>> >>>>>> >>>>>>Here's another question about Hsu's chess chip. I seem to recall reading some >>>>>>time ago that Hsu was considering a commercial release of his chip. Does anyone >>>>>>know anything more about this? If the chip were to become available, how could I >>>>>>use it in conjunction with a PC? would the fixed depth not be "out of sync" for >>>>>>the speed of, eg a Pentium 333Mhz if it was designed to work with a >>>>>>supercomputer, or can the fixed depth be adjusted to redo the balancing act in >>>>>>the new environment? If it were possible, I would be very interested in >>>>>>experimenting with this sort of hardware coupling. I assume that it would extend >>>>>>the depth to which a program could search by something like 4 extra plies within >>>>>>the same time. This would surely improve the strength of the PC ches programs >>>>>>quite a lot! >>>>>> >>>>>>Roberto >>>>> >>>>>You see it wrong. If you do 4 ply searches without hash etc, then >>>>>2.5 million drops quickly to say 300k nodes a second. >>>>> >>>>>So in fact you're playing against a kind of fritz5, which DOES search >>>>>all leafs fullwidth, which gives you some extra tactics, so commercial against >>>>>programs which are only tested at the same hardware and are only >>>>>busy with outbooking and trying to finish the game by means of tactics, >>>>>you beat with big numbers then, but it will play horrible. >>>>> >>>>>Vincent >>>> >>>>Hi Vincent, >>>> >>>>I'm not sure I understand - can you explain what yo mean. I do use hash tables - >>>>would not the Chip have access to my system's RAM? And When you say it would >>>>play horribly, why so? how could the chip make it play any worse than without >>>>the chip? >>>> >>>>Best wishes, >>>> >>>>Roberto >>> >>> >>>you have to ignore part of what vincent writes, because he is an "anti-deepblue" >>>person from way back. A couple of key points: >>> >>>1. no a chess processor could not see your RAM. But they do have their own >>>hash table memory. >>> >>>2. a PC program with one of these chips attached would be far and away stronger >>>than any existing computer chess program running on a PC. A chess board that >>>they used had 8 processors n it. A PC could use the same approach, and search >>>at about 20 million nodes per second. >>> >>>I pointed out to Vincent last week here that he was not doing well playing >>>against Crafty. He pointed out (factually, by the way) that Crafty has a >>>hardware advantage with the 4 processor ALR. But he's going to have to take >>>a stand on one side of the fence or the other. Either speed is important, and >>>his note about me having faster hardware does explain why crafty wins most of >>>the games played on ICC, or speed is not important and doesn't explain crafty's >>>win/lose ratio vs the various DiepX programs on ICC. >>> >>>I believe in a combination of speed and intelligent. DB has that. A PC-card >>>with one or more DB processors would *still* have that. And it would produce >>>unheard-of performance figures for a PC-based chess program. Ala' Deep Blue >>>Junior, which is basically a PC with a chess board (actually an IBM single-cpu >>>workstation with a VME-bus chess card, so don't start hoping to buy one of those >>>and plug it into your PCI bus. :) >>> >>>Bob >> >>You have to ignore most of what Bob writes about Deep Blue, as he is a >>known pro-deep-blue authority. >> >>As i pointed out a 'smart' program like deep blue can never do much >>knowledge within 10 clocks. This means that total depth of everything >>is 10 clocks. >> >>Also first Bob wrote that Deep Blue had 1000 adjustable parameters. >>As i pointed out 1000 adjustable parameters are hardly more than >>piece square tables (already taking 12*64=768 adjustable parameters). >> >>So when smoke about knowledge has been removed, we can go to >>the supposed speed of it. >> >>Then it appears that from the few printouts i saw of Deep Blue, >>that it just searched 11 or 12 ply. >> >>That's not much for a 200 million nodes a second program. >> >>Now when we consider that it has MASSES of disadvantages a normal >>chessprogram has at a general purpose processor, not to >>confuse with single chips, then we see after some calculations that >>those processors indeed are fast, but their practical speed compared to >>PC programs is a lot less. >> >>Still more than PC programs of nowadays, so no doubt it'll kick some butt >>in blitz, but i doubt whether normal users who likes to see some positional >>insight >>of a program as well will ever be happy with it. >> >>Further i like to point at the fact that the technology used for DB >>chessprocessors is very cheap. The salary of the PR people who were >>needed to organize the event needed probably eated up the biggest part >>of all that money, which IBM claims that the event Deep Blue-Kasparov took. >> >>Greetings, >>Vincent
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.