Author: KarinsDad
Date: 21:21:53 02/12/99
Go up one level in this thread
On February 12, 1999 at 17:32:35, Dann Corbit wrote: >On February 12, 1999 at 17:03:53, KarinsDad wrote: >[snip] >>But this is irrelevant. Isn't it? If you have a large enough sample set, the >>correct answer is correct, regardless of whether the computer thought it was >>correct for a different reason. When you actually have computers or humans play >>games, it doesn't matter why a move is choosen, just that given a large sample >>set of games, there is a win-draw-loss record against opponents of certain >>"strength" and hence, you have a rating accordingly. >Depends on what you want to measure. If you want to know the computer's >ability, you should measure whether the choice was accidental or not. Let's say >we are changing answers from true/false to a bit of a short answer type. In >other words, a monkey can get 50% right on push the true/false button but if >you ask why each time, it will drop to 0. The monkey does not know why. > Not necessarily true. Every year, SATs and GSATs are given which are multiple choice. Some people study like crazy and get better scores since they know questions from previous tests and how trick questions are introduced. Does this invalidate the testing? Of course not. If you had a large test suite (similar to the large number of questions on an SAT) and you change the suite every year or two, you would quite easily be able to evaluate the relative strength of a program. Would this give you an absolute strength? Of course not. Not even playing head to head and getting an ELO rating does that. It just approximates it. [snip] KarinsDad
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.