Author: Don Dailey
Date: 08:08:56 04/23/98
Go up one level in this thread
>Hi Don: >Interesting concept you have here: intransitivity and, implicitly, his >contrary, transitivity. Only I would like to add that both things maybe >works differently according to which level human oposition belongs. I >mean, a computer 100 point better than the average computers is a lot >stronger and above human average results against average computers than >the 100 points would indicate. And vice versa, if average computer gets >-let us say- only 15% of wins against GM, a computer 100 point stronger >in comparison with the average computer will not get a proportionally >better result against GM, but a lot less. This is just a guess based in >the premise that each pont computer win thought better programming means >increments of strenght that are relative to the kind of oposition they >get. So, the old problem of the relation between the Elo of a program >pitted against computer Vs Elo won against human is perhaps even more >complex that already seemed to be. >Fernando The main reason I bring up issues of transitivity at all is that most people blow it way out of proportion. I believe there is definitely some degree of itransitivity, if you measure the strength of programs among themselves you will get distorted ratings compared to humans. In other words the best computers will get overrated if the average computer is adjusted to be accurate. But lot's of people latch on to this idea of intransitivity and try to use it to explain everything. They imagine hundreds of rating points worth of intransitivity and I think this is unreasonable. Another setting in which this has happened is with speed chess. Some people imagine that certain programs play hundreds of points stronger at speed chess than at tournament levels. But it has always been so. It's always the very same programs that are believed to be especially good at speed chess even though computers have gotten over 100 X faster in a few short years. Wouldn't it make sense that the computers that were better at long time controls would be best at speed chess when computers get 10X faster? I know a programmer who believes his program would be the very best given an hour a move or so. If he is correct, then we should all watch out, his program will be unbeatable in 3 or 4 years or so. I've had long discussions with Larry Kaufman about this and he noted that most chess programmers think their programs are better (relative to other programs) with more depth. If Larry is right, then since they cannot all be better someone must be wrong! This isn't strictly intransitivity (which is a beats b, b beats c but a cannot beat c) but it's a related phenomenon. I think it's caused by a judgement defect in the human brain which we all have. Most of us make quick judgements based on little evidence. For instance if we we see ONE impressive win it's hard to be objective and believe it's possible that the loser may still be the better player. Intransivity exists among all players (humans and computers) because the rating system is based on some assumptions that are not strictly true. But I really believe it's rare for intransitivity to make up for more than a few rating points. To convince me of this you would have to show me a LOT of data. I can imagine someone now presenting me with an anecdote showing an example of some lower rated player winning a match convincingly. But anyone who does computer/computer testing should know that any short match (less than 100 games) can produce pretty lopsided results. One might imagine that programs that do significant pre-processing might exhibit a strong tendency to weaken (relative to others) with depth. I argue that this effect will be very small. If they play well at 5 second chess, they'll do reasonably well at tournament time too. The depth programs currently look are still pretty puny. I think pre-processing takes it toll but I can't say we are looking deep enough yet for this to amount to very much, and it does have plenty of advantages. I think there are still many great programs that do significant amounts of pre-proccesing. - Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.