Author: Bob Durrett
Date: 08:26:42 01/23/04
Go up one level in this thread
On January 23, 2004 at 10:12:00, Rolf Tueschen wrote: >We just had a little dispute about an old topic. When can we say that a prog is >better than another? How can we proceed to make sound arguments? > >Let me tell the story in fast mode. > >There was a test. I understand with 300 games or such. An incredibly high number >of games because often we have matches with onl 20 or 40 games. > >I understood further that on the base of a confidence intervall of 1-58 we have >95%. > >Now what I want to tell you, and this is undisputable statistical standard: > >if you get a value that is in the intervall, we cannot conclude that the >difference of the two progs is relevant or valid or call it what you want. It >makes no sense to argue with such "low" differences. They could be still be on >the base of chance. Now the distribution of chance is the Bell curve. Nothing >else. > >We had the debate with the SSDF list often enough. > >Two progs stand at the top. One is number one in the ranking. But is it really >stronger than prog number two??? > >The answer is easy. If the normal variation, this famous +- value in the SSDF >list is say +-40 points and the difference between progs is 35 points THEN we >are unable to conclude anything for sure. It could be that 1 is stronger than 2 >but also the contrary could be true. Only from values >40 on we have >"certainty", statistically, that a prog in that specific design is proven >stronger than another one. > >This is all so simply and trivial that it is satifying to be able to clarify. > >Have fun, > >Rolf > >P.S. > >I just want to correct a heavy mistake in a former posting. There it was said >for Elo differences that the difference of say 1 Elo point would be speaking for >a better strength of one prog over another and you needed so and so many gasmes >to prove that... - - this is total nonsense. There is _no_ way to conclude >anything out of an Elo difference of 1 point, no matter if you have 300 or >100000 games. The difference of 1 Elo point is meaningless. It's nonsense to >even think about such neccessary millions of games to "prove" that. Statistics >also has something to do with normal human sense. We would always take such a >difference for _equal_ strength. I enjoyed reading your bulletin, Rolf. : ) Unfortunately I admit not comprehending properly, but that is probably my oncoming senility. Of course, if you do not comprehend the following then you will have to admit to the possibility of your oncoming senility too! : ) : ) I often say to people: "There are no guarantees in this life." This could be paraphrased to "There is nothing absolutely certain in life." [I do not wish to discuss religion.] My point is that statistics is a helpful tool in man's pursuits but not absolute. I see statistics as being useful in several ways. First, it helps in the pursuit of making some sense out of a mess of data. Secondly, it provides a convenient way to express our expectations, providing such exotic terminology as "confidence," and the like. I see a tournament as being a tool too. It helps us to improve our odds of being on the right path. If we seek to determine who [or what] is best, who is second best, etc., then statistics provides us with useful tools to improve our chances of "getting it right." But there is no guarantee. It appears absolutely true that every tournament provides some information. The problem is in identifying that information, measuring it, and finding a good way to express or communicate that new information. This forum seems to have a problem coming to grips with the idea that there is useful information in every tournament. People seem "hung up" on the mandate to determine a "winner." That causes all kinds of misunderstandings. Well, so much for my "brilliant gems of wisdom." I'll try to do better next time. : ) Bob D.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.