Author: Joachim Rang

Date: 11:21:00 07/17/04

Go up one level in this thread

Hi Remi, that is exatly what I need right now, thank you very much. Unfortunately the MonteCarlo calculation takes really long if I feed in the result of two 400 games-gauntlets of two different version. I aborted after 10 minutes. Any idea how long it might take to compute on my Athlon xp 1540 Mhz? Is ther a way to make it a bit less accurate but quicker? regards Joachim On July 17, 2004 at 10:07:36, Rémi Coulom wrote: >Hi, > >I have made a new tool to estimate the likelihood that one program is better >than another, based on game results against the same opponents. You can download >it here: > >http://remi.coulom.free.fr/WhoIsBest.zip > >I have no evidence, but I expect its results to be more satisfactory than any >result based on Elo theory. This tool was very useful to me during the >preparation of the WCCC, to test differences between versions of TCB. I hope >some of you will find it useful too. > >Here is a sample output of MonteCarlo.exe: > > This program evaluates the likelihood that program A is better than >program B, based on the result of two matches played against the same >opponent (or set of opponents). The number of games played in each of >these matches does not have to be the same. If playing against a set >of opponents, the proportion of each opponent should be the same in each >match. > The likelihood is estimated by Bayesian inference, assuming an uniform >prior distribution of the probabilites of losing and winning. > The resulting integral is estimated with a Monte-Carlo method. It may >take a long time to converge when the number of games is large (>100). >The computation can be interrupted at any time with Ctrl-C. > >A wins = 3 >A losses = 4 >A draws = 5 > >B wins = 6 >B losses = 7 >B draws = 8 > >P(A>B) = 0.459296 (127000000 Iterations) > >Rémi

- Re: New tool to estimate the statistical significance of match results
**Rémi Coulom***13:28:26 07/17/04*- Re: New tool to estimate the statistical significance of match results
**Joachim Rang***13:43:56 07/17/04*- Re: New tool to estimate the statistical significance of match results
**Rémi Coulom***11:41:23 07/18/04*

- Re: New tool to estimate the statistical significance of match results

- Re: New tool to estimate the statistical significance of match results
- Re: New tool to estimate the statistical significance of match results
**Rémi Coulom***11:38:16 07/17/04*- Re: New tool to estimate the statistical significance of match results
**Robin Smith***12:52:04 07/17/04*- Ignore post above, Great idea! would be interested in a faster version
**Robin Smith***12:53:04 07/17/04*- Re: Ignore post above, Great idea! would be interested in a faster version
**Russell Reagan***14:35:50 07/17/04*- Re: Ignore post above, Great idea! would be interested in a faster version
**Rémi Coulom***23:49:09 07/17/04*

- Re: Ignore post above, Great idea! would be interested in a faster version

- Re: Ignore post above, Great idea! would be interested in a faster version

- Ignore post above, Great idea! would be interested in a faster version

- Re: New tool to estimate the statistical significance of match results

This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.