Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: New statistical analysis tool (can deal with longer gauntlet matches

Author: Joachim Rang

Date: 11:50:47 07/18/04

Go up one level in this thread


On July 18, 2004 at 14:40:01, Rémi Coulom wrote:

>Hi,
>
>I compiled a new version of the tool that I distributed yesterday. This one can
>deal with larger numbers of games. I tested that it works for a 6000-game match,
>so that should be enough for everyone. You can get it at this address:
>http://remi.coulom.free.fr/WhoIsBest.zip
>
>The only difference with yesterday's program is that it is compiled with gcc
>instead of msvc. the "long double" type is 80 bit in gcc and 64 in msvc. The
>additional accuracy is enough to solve underflow problems.
>
>Concerning Joachim's problem, here is the output of the program:
>
>  This program evaluates the likelihood that program A is better than
>program B, based on the result of two matches played against the same
>opponent (or set of opponents). The number of games played in each of
>these matches does not have to be the same. If playing against a set
>of opponents, the proportion of each opponent should be the same in each
>match.
>  The likelihood is estimated by Bayesian inference, assuming an uniform
>prior distribution of the probabilites of losing and winning.
>  The resulting integral is estimated with a Monte-Carlo method. It may
>take a long time to converge when the number of games is large (>100).
>The computation can be interrupted at any time with Ctrl-C.
>
>A wins   = 197
>A losses = 108
>A draws  = 95
>
>B wins   = 189
>B losses = 130
>B draws  = 81
>
>P(A>B) = 0.890434 (600000000 iterations)
>
>Rémi


thank you very much for this updated version.

regards Joachim



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.