Author: Dann Corbit
Date: 18:20:40 05/21/01
Go up one level in this thread
On May 21, 2001 at 21:08:08, Bruce Moreland wrote: > >On May 21, 2001 at 18:33:32, Ratko V Tomic wrote: > >>> I flip a coin 5 times and it comes up heads every time. >>> Does that mean the coin is biased? >> >>It is not as simple as a coin toss. If all you extract out >>of 100+ plies played in a game is the result (one of values >>1,0.5,0), then you're wasting 99.9.. percent of available >>information. You can play a single game against, say, a GM >>and you will know a few moves after the opening that you're >>dealing with a player much stronger than you. Or, playing >>against a complete novice, you can again easily guess that >>his/her rating is much lower than yours, even though not >>a single game has been finished, and according to your >>chess==coin_tossing theory, there is zero information >>available. >> >>We have also all seen quite accurate ratings of programs >>extracted after only few dozen next-move evaluations (e.g. >>with well calibrated test suites). A single game has often >>several times more positions that program has evaluated, >>therefore the information is there even after a single >>game to get the prgram's rating within a 50-100 points. >> >>The only model in which your coin tossing theory of chess >>is meaningful is a mindless rating estimator taking in >>only 1.58 bits of information per game (log2(3)), i.e. >>the final game outcome and nothing else. Looking >>through that kind of tiny pinhole anything you look at >>will look random and senseless. > >The opposite is also dangerous. You can watch a game where one program >completely dominates the other one, causing you to believe that you have enough >information for an accurate assessment, and yet in the next game the roles are >reversed. The recent match between Deep Junior and Deep Fritz (for instance). Or consider the recent game between Quark and ChessMaster. Quark has rapidly become a strong program, and yet CM is still stronger at this point. But if you look at that single game, CM was outplayed at every turn. >I think that there is a lot of information in a game, but I wouldn't trust >anyone to extract it properly. The observer is watching a chaotic sequence and >trying to extract meaning. A lot depends upon what the observer feels or >believes. If you tune your TV to an unused channel and watch the static, your >brain will create meaning. > >I don't believe in test suites as indicators of rating. The suites are created >by selecting positions that return good looking results for a bunch of programs. > Since they are designed to produce a specific result, it's hard for me to >understand why people praise any suite's predictive powers. > >It's like a magician who puts a bunch of colored balls in a bunch of colored >boxes, then covers his eyes and tells you which color box has which color ball >inside it. Of course he'd know which box contains which ball. He's the one who >put them in boxes. We can take any experiment down to the individual atoms and study the interactions between the elementary particles. Does it provide more information than: "The phenolthalien solution turned red" or "The phenolthalien solution turned blue" ? I don't think you can learn much of anything by looking at a single game. I remain thoroughly unconvinced, despite the many experts who say otherwise. And if you could learn something by looking at a single game, a trial of 500 games will likely bring out the same finding if it is correct. And if not, the wrong guess is likely to be exposed. Therefore, there is little to be learned from trying to make a judgement based upon a single observation. IMO-YMMV.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.