Author: Dann Corbit
Date: 12:09:50 10/20/98
Go up one level in this thread
On October 20, 1998 at 13:03:00, Amir Ban wrote: >On October 20, 1998 at 10:37:36, Nouveau wrote: > >>I don't understand these statistical stuff : I can't imagine a 12-8 result in a >>match between 2 GM with a conclusion like "They are very close in playing >>stregth". >> >>Why do we need hundreds, maybe thousands of games between computers to evaluate >>relative strength, when few dozens are more than needed for human GMs ? >> >>Jeff > > >I think this is a very good question that doesn't get enough attention. People >in computer chess tend to look at games played between computers as random >events. This, I find, is often puzzling to laymen, who (rightly) ask: What's >luck got to do with it ? > >But the other side of it is that we start of thinking of games in human chess as >random events too, which is a shaky conclusion. I'm aware of two big >differences: > >1. Computer programs are constant. Since they are constant, it is possible to >think of a program as coming with a rating attached, which is fixed but unknown. >Every game we play is a sampling of its strength, and if we play enough games, >we will get a good estimate of its true strength (which really never changes). I think that these programs are not nearly as constant as people assume. The temperature changes a few degrees, and the program happens to exactly manage one ply deeper -- or makes one ply less. The computer gets an IRQ because of a disk read ecc error during play, reducing the ability. A computer learns positions as it plays, and hence plays the exact same starting sequence at the exact same time controls differently the second time. I don't think that computers are necessarily even more deterministic than humans. If we play Fritz verses Hirarcs, ten matches of 16 games at 40/2hrs, will we expect the exact same results? I certainly do not. > It is not possible even in theory to make this assumption for humans. They >change all the time, and by the time they have played enough games to change >their rating, their true strength is already different. Since we don't know what >we are measuring, talking about statistics is meaningless. I think that they are meaningful, with exactly the same value as against machines. We can consider a huge number of games by a top player and, using ELO's formula, find the probability that they will defeat another player that has played a huge number of games. If there were no sense to the measure, then 1500 players would win major tournaments once in a while. Instead, we find that the best players present tend to win. Now, the reasons that they don't win really do not matter. Whether it is a headache, or a bad night's sleep, or they 'tied one on' the night before -- whatever. Just as we do not care 'why' the chess playing machine won or lost when we are taking statistical measurements, we only want to know how well we can predict. I suspect that a careful probability study will show that matches between computers and matches between people have about the same amount of predictability. In other words, Kasparov will not lose frequently to inferior players, nor will a great chess program lose frequently to an inferior program on inferior hardware. >2. Unlike computer games, which often start from a truely random opening line, >there is nothing random about games between humans, and everything they do is >100% intentional. Humans do not stumble into openings. They have total recall of >past games, their own and everybody else's, full knowledge of their opponent, >etc. Indeed, I think the way humans play and computers is completely different. Despite the different methods, there is still a chaotic element. A great player can make a blunder. Computer programs blunder also, for that matter. > It is of course possible in principle for computers to have 100% >intentionality like humans. > > I don't think humans would agree as describing games between them as random >events. They would be more inclined to quote luck (if they do) by saying "they >had a bad day", or "they were in top form", which is not the same thing. I think >a GM would regard the sort of probability calculations done here as idiotic and >inappropriate if applied to his latest result in a tournament, or to his career >(Did Kasparov win enough games to justify his standing as the world's best, or >should we declare his career statistically insignificant and ask him to play a >few more before we decide ?) I think that all events are random events, to some degree. The migration of birds, what I choose for breakfast, what I am typing right now, whether this message will even reach the newsgroup. There is an element of chaos in everything. Einstein probably would not be too happy with that statement, and he was certainly a lot smarter than I am. But I think that generalizations are exactly that. Not only that, but Kasparov is not necessarily the best player in the world. Just as a 'world champion' soccer team may not be the best in the world. They are simply proclaimed as champion. But if they had to play the team they beat to win the championship 100 times would they win all 100? Or 1000 games. Or one million. If not, then one of the loss(es) that occurred could have happened during the championship match. Every so often, there may be a match which determines the world champion. What if the truly best player was simply not up to form that week? The inferior player would be proclaimed champion. I don't believe that there really exists a world champion at anything. They are only proclaimed as such. We can never know with 100% certainty that anything is really the penultimate.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.