Author: Don Dailey
Date: 10:15:47 09/06/98
Go up one level in this thread
On September 06, 1998 at 10:18:46, Enrique Irazoqui wrote: >On September 05, 1998 at 18:10:01, Don Dailey wrote: > >>I have always been interested in this question of intransitivity >>between humans and chess programs. A lot of people claim, almost >>always based on some anecdotal evidence, that various programs >>are much better or weaker against humans or other programs and >>that huge intrasitivities exist. My own thinking is that this >>is minor, but I know I'll get a lot of disagreement here. >> >>I can't tell you how many times I have heard this type of >>conversation: "I played a game with program >>X and got crushed, then I played a game with program Y and >>won easily. But when I played the two programs together, >>program Y demolished program X." From a measly 3 games >>can you come to the conclusion that program X is not very >>strong against other programs but is "crushing" against >>humans? >> >>But once you form a conclusion, then you start noticing the >>events that reinforce your conlusion and you minimize the >>events that do not. Usually, the opinion propogates to other >>people if it gets stated enough times. >> >>Here are the ones I have heard but have serious doubts whether >>they are true, or at least crystal clear: >> >> 1. Novag machines a much better against people but no good >> against other programs. >> >> 2. Genius is not very good against people but crushes other >> computers. >> >> 3. Genius is particularly good aginst people, but just so-so >> against other programs (yes, I've heard both cases stated >> as facts.) >> >> 4. Deep Blue will crush any computer but is not much better >> that micro programs against humans. >> >> 5. Any kind of forward prunning or selectivity will help a >> lot against humans but is not so good against computers. >> >> 6. Same as 5 but reversed. >> >> >>- Don > >Also heard (often): fast searchers will do better against programs, while >knowledgeable programs will do better against people. That is a good one! I forgot to mention this one, I've heard it a lot of times too. I think this one is based on the idea that computers ALREADY outsearch humans, but need to play "catch up" positionally. It is easy to imagine someone reasoning this way and thus drawing this conclusion. I think this kind of reasoning is a fallacy although I have no idea if the original assertion is true or false. Except I do believe that either way it's probably quite minor. I really think we have a long way to go in terms of how to think about these things. We have not developed the correct terminolgy to even talk about it. For instance: What is positional play? What is tactics? What is a combination? You will get a different answer from each one and the answer will not be one that can be used in any kind of proof. Shall we try it on the group? What is a combination? Give an answer that has no ambiguity whatsoever. I have never seen one in any book, but I've seen many attempts. You may be able to come up with a strict definition of what a combination is, but I'll be many will disagree about what this definition should be. Your definition will probably not match peoples perception of what they think a combination is. >I have seen no proof of any of this. I have seen no proof either of the >supposedly huge difference between comp-comp and man-comp performance. > >Taking Aegon 91-97, not 40:2 but quite close to it, we have the following >statistics (taken from Rebel's home page): > > TOP 12 FOR THE MAIN-COMPUTER PROGRAMMES > 1991-1997 > ---------------------------------------------------------------- > 1. Rebel +21 = 8 - 7 25/36 = 69.4% > 2. Chess Genius +15 = 9 - 6 19½/30 = 65.0% > 3. Chessica (Fritz) + 9 = 5 - 4 11½/18 = 63.8% > 4. Hiarcs +16 = 6 - 8 19/30 = 63.3% > 5. The King (Chessmaster) +22 =14 -12 29/48 = 60.4% > 6. M-Chess Pro +26 = 5 -17 28½/48 = 59.3% > 7. Chessmaster (The King) + 8 = 5 - 5 10½/18 = 58.3% > 8. Virtual Chess + 9 = 3 - 6 10½/18 = 58.3% > 9. Fritz +14 =13 - 9 20½/36 = 56.9% > 10. Quest (Fritz) +15 =11 -10 20½/36 = 56.9% > 11. Kallisto +17 = 6 -13 20/36 = 55.5% > 12. Nimzo +12 = 6 -12 15/30 = 50.0% > >Knowledgeable and fast searchers seem perfectly mixed in. > >Taking only the slow man-machine games played since 1996 (Aegon 96/97 + >Mchess-Efimov + Rebel-Anand), we have the following: > > Average Elo >1996/1998 Games of Opponents Performance Elo >Nimzo 12 2402 71% 2570 >Kallisto 12 2359 71% 2527 >Rebel 20 2371 68% 2515 >Mchess 18 2368 64% 2480 >Zarkov 12 2320 67% 2456 >King 12 2259 71% 2427 >Morsch 30 2306 63% 2410 >Virtual 12 2367 54% 2399 >Hiarcs 12 2212 67% 2348 >Wchess 12 2181 63% 2285 >Genius 24 2203 58% 2267 >CST 12 2175 54% 2207 >Shredder 12 2170 46% 2138 > >(Morsch = Fritz+Quest+Chessica) > >Few games to be conclusive, but again knowledge vs. speed doesn't seem to make >the difference. In fact, I am unable to see any pattern in there that can >support the opinions you mention above. > >I guess in most cases people base their opinions on subjective impressions, like >having an easier time playing against a passive program than an hyperactive one, >which makes them say "Mchess is much stronger than Genius against people", etc, >while someone else can say just the opposite. > >My 2 cents out of common sense (but who trusts common sense?): it might (?) make >sense to believe that an aggressive style (opening up the position, going for >tactics) will improve the performance of a program against people. But even if >this is true, it might make the program weaker overall. I mean, performance >against people is not the only measure of strength. And in the games above, the >most aggressive programs, like Mchess, the King and CST, are not the ones that >did best... So, in this regard it seems we know next to nothing. > >Enrique Those are great observations. I thought I was the only one who felt this way. I want to point out that I don't believe it is impossible to have this behavior, (I can by construction create 3 programs with intransitive behavior) I just don't think it's really happening very strongly with modern programs. We often (as humans) see patterns that do not exist. - Don
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.