Author: Dave Gomboc
Date: 19:00:45 09/09/02
Go up one level in this thread
On September 09, 2002 at 18:31:32, martin fierz wrote: >On September 09, 2002 at 04:58:21, Ingo Althofer wrote: > >>Yesterday I found the homepage of Martin Fierz, and on this his interesting >>report on the computer Checkers tournament that had been played in Las Vegas >>some weeks ago. The article is nicely written and worth reading, even for normal >>computer chess enthusiasts. See www.fierz.ch/vegas.htm >> >>However, there is a point where I disagree with the author. In a few remarks - >>and maybe mainly between lines - the reader gets the impression that the author >>has not a very high opinion of Jonathan Schaeffer's work in the Chinook project. > >hi ingo! > >you have to take the report as what it is: a very personal account of my >experience in las vegas. it is not a scientific paper, more a casual write-down >of my impressions during and right after the tournament. personally, i rather >read honest accounts of what's going on in people's brains than reading some >kind of censored version after the fact, and that's why i wrote it this way. > >well, i guess i should clarify this thing about chinook that you read between >the lines: there is no doubt for me that chinook was a good program, and that >it's authors did a great job, especially in computing the endgame databases on >hardware which wasn't really up to the job. i don't mean to belittle *anything* >they have done in making chinook into what it was. Frankly, that statement's inconsistent with your statement about Chinook's title not being worth much -- a comment that may be your honest opinion but is plainly rude. >the situation with chinook is very similar to what computer chess has with deep >blue: a program which was running on hardware so much faster than anybody else, >that you simply cannot tell whether the program itself was also really good, or >if it was mainly the hardware. personally, i find it sad that schaeffer & co did >not come to play in las vegas, as it would have been really interesting to see >how good chinook was as program, not as hardware. That hardware wasn't so fast. It was fast for the time, but that's about it. Today's machines blow right by them. For instance, if you analyze the moves played, you'd have to give an engine today a fraction of a second (I'm guessing, I didn't work it out exactly) to search to get the same amount of computation. I know that Jonathan worked hard on Chinook "as a program" back in those days. He didn't have big hardware to run it on all of the time -- just like Bob Hyatt didn't have a Cray at his personal disposal. >but yes, as you read betwen the lines, i do not have a terrible lot of respect >for what the chinook team did *after* winning the man-machine title. >1) they published their endgame database with access code which is so terribly >bad that you can hardly understand it. it does not work for the 8-piece db. so >they *must* have different access code of their own, which also probably is much >cleaner and faster, and they don't publish it. why? >if you don't believe what i say about that access code, just look at it... you >talk about "small-mindedness about user-unfriendliness". it's not about >"unfriendly". it's about "unusable". and about the fact that ed gilbert lost his >chance to win this championship because they didn't publish the better version >of their access code. imagine eugene giving out a version of his tablebase >access code which is both inefficient and unable to handle anything but 4-piece >EGTBs, when he really had a 4x faster version at home which could handle >everything. would chess programmers be happy about that? It seems that you're not aware that that code has been on the web for roughly a _decade_. It was written long before 8-piece databases even existed. It is unreasonable to insist that someone who puts some code up on a web page years ago has to update it and maintain it in lieu of conducting new research. Furthermore, it's well-understood that open source code doesn't come with a warranty: you're just looking a gift horse in the mouth. That's rude too. >2) schaeffer said on the phone to our referee, mac banks (who is a person i >respect, and who i think incapable of lying about this): "why aren't the best >programs playing in las vegas?". i don't know what he said this for, but yes, i >do take offense at such statements :-) I don't know about any of this. What I do know is that your report says that one possible contestant felt that the tournament format was unsatisfactory. Indeed, it apparently was so unsatisfactory to them that they chose not to compete! Perhaps he was asking about that program? >3) schaeffer is continuoulsy dodging the question whether chinook is now retired >or not. officially it is, but he says he will have to think about defending his >title. either you're retired or you're not. again, he said something on the >phone which i can't follow: "chinook is retired. but as a scientist, i'd like to >see how it does against today's programs". if that is really what he wanted, he >should have played in las vegas. with the chance to play all serious other >programs. I don't speak for Jonathan, but I know that he is a busy person and cannot just do whatever he wants whenever he feels like. He has research responsibilities, he has teaching responsibilities, he has business responsibilities, and he has family responsibilities. If your quote is accurate, I'd say it sounds like he doesn't have the time to get into the daily grind of updating this book line, tweaking that evaluation term, et cetera, but he would like to see how stuff he wrote almost a decade(!) ago stacks up against newer programs. >>And in my mind it is also ok when in the forthcoming title match Chinook as the >>defender will keep its title when the match ends in a draw. Chinook did its job >>years before the others did, and therefore they deserve this advantage. >i disagree with you here. >1) doing something first is not the same as doing something best. a title should >reflect the fact that a program is the best, and not that it was there first >IMO. computer chess world championships are organized in a much more democratic >way. all organizations which have "defending champions" run into trouble because >of this, all the time. chess. checkers. boxing. you name it! I can't see why you're at all worried about it. Frankly, the only remaining goal left in checkers is to solve the game. If you want a place in history, go for that. If not, then forget about checkers. :-) >2) if you play a match of checkers against a program like nemesis or kingsrow, >which have great opening books, you will very likely not win a single game in >100. the advantage of having drawing odds under those circumstances is that you >are virtually guaranteed to keep your title. IMO that is just not right. >these programs are so good, that you could take one of them, and have it play a >match against itself in a version running on a high-end PC, and the other on >hardware half as good. the result would most likely be N draws, even for N=100. >although one version is *objectively* much better than the other, because it >always finds the best move in half the time. would you really want a program >which is objectively worse than another to be called "champion?" A challenger has to be better than a champion to replace it. If Chinook, written a _decade_ ago, is still good enough that one of today's programs would not be able to defeat it, then why should it not be the champion? >finally, i have a contribution to make to the discussion "was chinook better >than tinsley?", which started somewhere else in this thread, with bob saying >tinsley was certain he would lose the match. of course, we can never answer the >question, so it's all speculation, but let me give you my version of the >speculation: >the follow-up match chinook-lafferty, ended with +1=18-1. if you look at the >games, you will find that chinook made two mistakes in 20 games: it lost a game >which was a draw, and it drew a game where it had a win. And the follow-up match to that was that Chinook defeated Lafferty. And Chinook never was in trouble against Ron King, always winning or drawing. And Chinook also beat the top correspondence human player, by a massive score (something like +7 =13 -0). So Chinook was pretty good, even though it made mistakes. I will remind you again that Chinook played in the days of single and double-digit megahertz machines. Now we have quadruple-digit megahertz. I doubt it would make those mistakes that you refer to on today's hardware. >my personal 2 speculative conclusions from this are: >1) if lafferty was able to draw a match against chinook, it is well possible >that tinlsey would have managed that too. it has been said that lafferty played >very conservatively, trying to draw all the time. tinsley was known for his >cautious play - he would probably have had a similar approach to the match. >note that i'm dodging the question of "was chinook better than tinsley?". i'm >just saying, it is possible that tinsley could have drawn a match, even if he >was maybe already a bit weaker than chinook - drawing odds are just HUGE in >top-level checkers... Tinsley was a great player. Sure, it was possible. >2) chinook made two serious (result-changing) mistakes in 20 games. it was not >as close to perfection as people generally think at the time it played that >match. and there were errors in it's endgame database too... Please refer to above re: hardware. >aloha > martin Dave
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.