Computer Chess Club Archives




Subject: Re: Proving something is better

Author: Peter Fendrich

Date: 05:59:19 12/27/02

Go up one level in this thread

On December 24, 2002 at 07:04:45, Uri Blass wrote:

>On December 23, 2002 at 12:14:57, Peter Fendrich wrote:
>>On December 23, 2002 at 09:10:59, Rémi Coulom wrote:
>>>On December 23, 2002 at 06:31:57, Peter Fendrich wrote:
>>>>I don't get the same results as you:
>>>>(all the sample values of Pw, Pd, Trin and cumTrin are the same)
>>>>          Mine         Yours
>>>>4 6 0 :   0.2745363   0.2745533
>>>>4 6 1 :   0.274543    0.2747703
>>>>4 6 10:   0.2740947   0.2773552
>>>>I'm not sure what's happening here but as you say, the Monte Carlo method
>>>>doesn't give exact values.
>>>>I thought that the program could be reliable to 3 decimals but maybe not...
>>>>However, if I'm right about draws, the prob would slowly move towards 0.5 when
>>>>the number of draws increases. I continued up to 500 draws and got the
>>>>4 6 100:  0.2758096
>>>>4 6 200:  0.2820387
>>>>4 6 300:  0.2906555
>>>>4 6 400:  0.3076096
>>>>4 6 500:  0.3273436
>>>That is because the Monte Carlo method is inaccurate: think about the x^n
>>>function, x varying between 0 and 1: when n grows large, the function has an
>>>extremely thin peak, that is very difficult to integrate accurately with a Monte
>>>Carlo method.
>>>I am totally certain about this, because I first started by implementing a
>>>program that calculated the big trinomial integrals. I wrote a small polynom
>>>library based on the GNU multiprecision library so that the _exact_ value was
>>>found. That's how I noticed that the result does not depend on draws, and went
>>>further in the calculations.
>>Yes, that's quite possible. What's funny is that I repeatably get these kind of
>>results for other combinations of win/lose. I have a sligth hunch about what
>>might happen. Let's come back to that later.
>>>>The probability for A neatly grows with more draws.
>>>>OTOH I can't argue against your formulas. Give me some more time.
>>>>Could you please elaborate the first formula in section "3 Draws do nout Count".
>>>>How do explain 1 - p0 - p0.5 = 1 - u + up0.5 - p0.5?
>>>>I'm a bit interested in the term up0.5 that seems to be superfluous.
>>>u is defined by p0 = u * (1 - p0.5). If you replace p0 by u(1 - p0.5) in
>>>1 - p0 - p0.5, the you get 1 - u + up0.5 - p0.5
>>Ok, u is a variable and nothing else...
>>Then it's prefectly clear that your math is allright. The connection to the
>>rating system is not as close as I, from the beginning, took for granted. It's
>>not the probability that is changing it is the interval describing the bell
>>curve, surronding 0.5 that is shrinking when the number of draws increases. I
>>had to get a short discussion with another person in order to get really
>>convinced about this strange fact.
>>That gave rise to a new question.
>>Maybe he will post something himself otherwise I will come back with an better
>>explanantion of what's folling ...
>>The new question was about the problem formulation. The issue is not right
>>formulated from the beginning ("is A *better* than B" given a certain match
>>result). Draws contains some information and when we have a question formulated
>>in a way that makes draws not to be included, we will not get full quality of
>>the conclusion. The area of interpreting match results is interesting and not as
>>evident one might think.
>>I'll return later on...
>If you want to give an interval for the difference in rating that you want to be
>sure in 95% that the real difference in rating is in the interval then the
>number of draws is relevant.

Yes, indeed but than we also need a enough number of games in order to use the
Normal distribution as an approximation of the Trinomial one.

The whole discussion started with "what can be said about match results with a
small number of games".


This page took 0.02 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.