Author: Dann Corbit
Date: 15:43:59 03/27/04
Go up one level in this thread
On March 27, 2004 at 10:44:53, Pat King wrote: >On March 26, 2004 at 19:25:28, Dann Corbit wrote: > >>It is a mistake to trust a 10/0 result, and especially >>against a single opponent >>if you are trying to judge strength. > >It is your belief that it is a mistake to trust a 10-0 result. Statistical >theory, however, disagrees with you. Guess who I trust ;) You know it is stronger. But you have no idea how much stronger. If that is all you care about then you are done. >You do have a good point, however, on what conclusion one can draw from this >result. I would conclude that engine A will continue to beat engine B with a >high degree of confidence. That says nothing about how A and B would do against >engines C through Z. 30 measurements are needed for reliable statstical inference. You can't even use the T-test or some other estimatrs with less than 30 measurements (or you shouldn't because the result is invalid). See (for instance): http://math.usask.ca/~szafron/STATS244_chapter9_notes.pdf http://www.stat.colostate.edu/~hess/ST101/Notes/ Without at least 30 observations, the quality of the computed answer is so low as to be nearly useless. Is it better? Quite likely. How much better? You haven't an inkling yet.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.