Author: Bob Durrett
Date: 19:04:49 10/25/02
Go up one level in this thread
On October 25, 2002 at 00:10:20, Aloisio Ponti Lopes wrote: <snip> >A. Ponti The idea that it would be useful to include reference chess programs in the tournament seems to have some merit, although this would have to be carefully controlled to assure accuracy and usefulness. To be more specific: If a moderately large chess-playing-program tournament included a small number of earlier version chess programs, with the SAME opening books, parameter settings, and computer hardware as were used the first time SSDF tested those engines, then the performance of those "reference" programs would be known fairly well. The performances of these "reference" programs could be used [to some extent] to calibrate the rest of the tournament, as noted by someone else in an earlier bulletin. The presence of such "reference engines" might help the analysts to make more sense out of the tournament results. _ _ _ _ _ _ _ _ _ While on this topic, I would like to make a small point about the value of "small tournaments" and "small matches." Chess games are essentially totally different from tosses of a coin, or of dice. After a game is completed, it is possible to exhaustively analyze each and every move in each game. If the times for the moves are recorded, then that information can be used too. [It would also be possible to monitor internal engine parameters or variable values, and this maybe should be done in the future.] The amount of information available is ***CONSIDERABLY*** more than that of the final tournament score. Who would deliberately ignore all available information? Simple statistics based solely on win/draw/loss should NOT be used as the primary method for evaluation of tournaments or of the performances of the competing chess programs. The eight game Kramnik vs DF match produced FAR more information than would be obtained by tossing a coin eight times, for example. [Not even a six-sided die would suffice.] The statistical significance of those eight games cannot be computed without taking into account ALL of the information which the match produced. SUMMARY: The comparison of chess tournaments with coin tosses is simply inappropriate!!!!!!!!!! Bob D.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.