Author: Dann Corbit
Date: 12:16:04 10/22/98
Go up one level in this thread
On October 22, 1998 at 12:21:50, Fernando Villegas wrote: [snip] >You are right, but let me add just another thing. We not only get results in a >10 games match, but also we get moves and we can take a look at them. Loking at >them sometime just a game suffices to see which player is stronger. You can be >faced before two masked players and I am sure you will identify very soon who is >the strongest and better player just because the quaslity of the moves, IF a >perceptible difference exist, of course. In that sense I tend to believe what >Thorten said many times about his capacity to measure a program strenght just >looking his moves is maybe an exagerated sentence, but there is a point of truth >in that. Precisely because results of a game are not random but obey to the >intrinsic qualities of the player, we should not take this just as the >experiment with the coin. In other words: a program is not stronger that another >BECAUSE got 85% of the points, but got 85% or in average will get that number of >wins because play better moves. Sometimes I can see brilliant moves, but being a patzer, often a move that looks stupid to me is simply over my head. I agree that there is some merit to this sort of analysis, but all that it means is that an expert will likely see which one is stronger before the math reveals it. On the other hand, a computer can also blunder into a brilliant move, making it look much stronger than it really is. >Of course, this is increasingly difficult to measure as much programs are near >to each other, but adding statistics results to qualitative observation of moves >i am sure we can get some very relevants conclusion not a lot beyond ten games. I agree also that we can begin to make predictions early. The validity of the predictions becomes stronger with greater numbers of matches. In most cases, we will find that repeat matches confirm our hypothesis. But this will clearly not always be the case. Let's consider another example. There are NN chess playing programs that exist. Now, quite frankly, they stink, as I can usually beat them without too much trouble (this could become a formal guideline -- "Can Dann Corbit beat it most of the time? If so, then it stinks.") BUT, these systems improve with each game. It takes hundreds of thousands of games before they can play with any strength at all. With trillions of games, they might possibly become world-beaters. In other words, these programs are not static objects, but change over time. There are lots of chess programs that learn as they play, with Crafty being one obvious example. So when we look at these experiments, we must recognize that sometimes we are dealing with a moving target. Really, all observations are that way to some degree. For instance, if I measure a metal bar, and find that it is 290.32cm, I might measure it later and find that it is 290.35cm. This might not be a measurement error, as temperature change can actually change the length. I think that quite likely some of the predictions about this program or that program *may* be correct. But I don't think it is valid to state these opinions as factual. I brought the whole thing up because repeatedly I see wild conjectures based upon a single observation.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.