Author: Don Dailey
Date: 13:10:01 08/25/98
Go up one level in this thread
On August 24, 1998 at 16:38:37, Mark Young wrote: >On August 24, 1998 at 15:56:27, Robert Hyatt wrote: > >>On August 23, 1998 at 19:45:20, Don Dailey wrote: >> >>>On August 21, 1998 at 23:55:07, blass uri wrote: >>> >>>>I think it can be a good idea to do a tournament of programs when >>>>every game between 2 computer programs will be replayed 3 times: >>>>1 time when both program have an opening book. >>>>1 time when only white has an opening book. >>>>and 1 time when only black has an opening book. >>>> >>>>By this tournament we can estimate the value of the books opening in elo. >>>> >>>>Uri >>> >>> >>>A reasonable idea. I have another idea for those situations where you >>>want to play long matches between just 2 machines to minimize the >>>effect of killer books. It's certainly not a new idea but a reasonable >>>one I think. You simply determine in advance how deep you can go in >>>the openings and then not allow duplicate games beyond this point. For >>>instance if you decide 8 moves provide adequate variety, then the >>>first time another game duplicates within the first 8 moves, (even if >>>the result is different) then you throw out the 2nd game and start over >>>until you get another unique opening. An opponent can still optimize >>>but this becomes much more difficult to do and the opponent program will >>>have to take more chances to do it right like having to put weaker moves >>>in his book to bait a specific opponent. >>> >>>Another similar idea is to decide in advance how many games should be >>>in the match (n) and play 2n games. Use the above idea but determine >>>what number of moves is just big enough to not exclude too many games. >>>This has the merit of not having to figure this out in advance but >>>requires much more testing overhead. >>> >>>- Don >> >> >>I have another alternative: How about we just continue to work on >>book-learning, which is still fairly infantile in the way things work, and >>let the programs fend for themselves? IE (and speaking only for myelsf here) >>my goal is to have a program that can stand by itself, against any sort of >>pranks you might throw at it... trying to repeat games... trying to use pre- >>calculated killer book lines... trying to coax it into dubious positions by >>offering a gambit pawn... >> >>It is too easy to find ways for a human to finagle a program so that it doesn't >>get burned. I find it *much* more interesting to finagle the program so that it >>finagles itself and self-avoids these silly messes. >> >I agree. I don t understand people s needs to test only the engine. When you >play a chess program, you play the whole program. And in my opinion the opening >play is a big percentage of any programs overall strength. And the sooner we >come to realize this, and stop treating it like an ugly step child, the sooner >we can address this area of chess programming that has been neglected by so >many. You may not fully understand the issues if you don't understand peoples need to test only the engine. I'll explain why this may be important to people: Years ago a master friend of mine wanted a chess computer, but was very concerned about how well it played the endings. He said he didn't care if it played weaker overall chess if had a better endgame. That is what HE wanted in a chess computer. That was his need. Other people care about how well a program will play a position you give it, they don't care so much about traps and swindles in the opening book or how vast or clever the encylopedia is. This is certainly legitimate. I disagree with those who say it has nothing to do with chess strength, it certainly does. But it's VERY hard for me to disagree with those who simply prefer a stronger evaluation engine to a better more integrated book. That is their choice. At one time I had a big problem with the so called "cooked books", I no longer worry about this. In fact I encourage it and think it is a completely fair and legitimate thing to do. Also I view anything you do in this area to be a strengthing of your chess program. But give me a choice between two programs that are equal in strength, but one has a much better and more integrated book, I will take the one with the weaker book every time simply because my preferences would be biased toward the program I felt was less dependent on the quality of the book. As far as doing studies to find out how well programs play without their books, or to minimize the effects of the books, I have to say I think this is very interesting! I concede that this will not measure a programs overall strength, but those who want to do these tests are looking for the same program qualities I would like to have in a chess program. - Don >>There are problems, because book-learning is not real good (at least my >>version) since a book can have millions of games in it, and you really ought >>to play each game once to confirm that it is playable or not... and that is >>*really* impossible... Someone suggested that the "crafty world" might do this >>by setting up each crafty clone to play only one specific opening and become a >>"expert" in it (since learned data is sharable with the import facility in >>Crafty). And this might be a solution... one only plays the French as black >>in response to 1. e4, and it would not take forever to play the various book >>lines and learn what is good and what is bad. >> >>But I like the "automatic" approach, rather than trying tournaments with no >>books, limited books, identical books, etc. Something tells me that Genius >>would "die" without the book it normally uses, because it has some lines that >>it handles very well, while other programs would probably not be happy. It >>doesn't make a lot of sense to spend time trying to test "parts" of a program, >>because it is really a complete entity, that includes the engine, the book, the >>endgame databases, learning information, and whatever else is part of the >>"thing". >> >>Also it seems more satisfying to solve something once, than to have to >>continually solve it over and over manually...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.