Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Statistics and Test results

Author: Rick Bischoff

Date: 15:40:14 10/07/04

Go up one level in this thread


>>You set your criteria before hand.  i.e., do you want to be
>>95% confident that you have the right answer? 99% confident?  Then, you decide
>>on the proper test to use and the sample size.
>
>Sorry, you can't decide the confidence level before you do the statistical test.
>It's output from the statistical test.

No, that is faulty; you can't decide on the confidence INTERVAL (for cetain
types of tests) until you do the test, but the level should be decided before
the test.

Your test, in a sense, was doing exactly this-- you said yourself, if the
variable "t" is outside of the interval (-2,2) you can be 95% sure that the
means were different.

Anyway, the point is moot since the basic assumptions are flawed.

1. You drop draw scores-- you should be either rolling them into wins or losses
or using a multinomial distribution model.
2. You did not state that which distribution you believed the number of wins -
number of losses to follow.
2a. Your sample is not random, although if this continues to be a picking point,
I will drop it as the rest of my claims are valid and can't be refuted.
3. Your transformation of variables and your test on the statistic "t" is not
valid-- you might be assuming a normal model, but I can't know that since you
didn't say so.  Nevertheless, even if you did, I can show you that your
transformations do not lead to a distribution that can be safetly approximated
by the normal distribution.



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.