Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: How many games are needed to find out which program is stronger?

Author: Dann Corbit

Date: 15:21:45 09/03/99

Go up one level in this thread


On September 03, 1999 at 17:59:46, Heiko Mikala wrote:
>On September 03, 1999 at 02:59:26, Dave Gomboc wrote:
>>Crack open an intro to statistics book.
>Dear Dave!
>Thank you very much for this tip and your always friendly replies to my posts.
>
>If you should ever feel the desire to find out more about me, a good starting
>point might be to ask me about my education and my profession.
What is your education and profession?  You have made me curious because my
degree is in Numerical Analysis.

>Hint: I may have "opened" more mathematical books than some other people have
>ever dreamt of.
I have at least a dozen books on special functions.  How's that for perverse?

>Respectfully,
>
>Heiko =:-(
I think you want the mathematical answer, then since you have hinted that you
are a mathematician.  I am guessing that your degree is in pure math, so the
answer is:
An infinite number of games is needed to be absolutely certain that program A is
better (or equal or worse) than program B.

A lesser number of games will determine the same outcomes with less and less
degrees of certainty.  I think what Dave was hinting at is that it is a fairly
standard statistical problem.  Probably most statistical texts will have some
sort of discussion on this sort of test.
E.g. :
Hypothesis: "The Douglass Fir's average height is greater than that of the
Ponderosa Pine"
How would you attempt to prove/disprove this hypothesis?

The best description is probably ELO's book itself (as a wild guess, since I
don't have a copy, but I do know that he talks about games and probability and
derives a boatload of formulae).

HTH

P.S.
Dave wasn't grumpy.  He's a grad student. ;-)



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.