Author: José Carlos
Date: 09:53:35 01/03/01
Go up one level in this thread
On January 03, 2001 at 12:36:51, Christophe Theron wrote: >On January 03, 2001 at 09:52:06, José Carlos wrote: > >> Lately, people have been talking here about significant results. I'm not >>really sure if probabilistic calculus is appropiate here, because chess games >>are not stocastic events. >> So, I suggest an experiment to mesure the probabilistic noise: >> >> -chose a random program and make it play itself. >> -write down the result after 10 games, 50 games, 100 games... >> >> It should tend to be an even result, and it would be possible to know how many >>games are needed to get a result with a certain degree of confidence. >> If we try this for several programs, and the results are similar, we can draw >>a conclusion, in comparison with pure probabilistic calculus. >> >> Does this idea make sense, or am I still sleeping? :) >> >> José C. > > > >I have done this experiment with Chess Tiger with fixed openings and reversing >the colors for each opening, on a large number of openings. If you use the same program and the same openings, what you're measuring is the effect of the program's randomness on the result. But, as normal games aren't played with fixed openings, there's still another thing to measure: book randomness effect on the results of a match. I guess that, with the same program and the same openings, the results would be pretty even from the begining. But, what I'm suggesting is, as in real games, let the books do their work and let the program play itself. The reason why I chose the same program is because, that way, I'm sure that, for a given number of games, the result must be very even. An example: if with this experiment we see that, after 1000 games, the result is 600-400, we'll sadly have to say that 1000 games are not enough, but if we see that, after 100 games, the result is 52-48, then we can safely say that 100 games give a rather certain result. Of course, these examples are not absolutely correct, but show my point. >This experiment and the results I have got is the reason why I say all the time >that statistical significance is very important. Yes, that's my point too. That's why I suggest a way to measure what is the "degree of noise" depending of number of games. >When you see a program beating itself 10-4, you begin to understand what I mean. That's what I wanted to test. > Christophe José C.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.