Author: Uri Blass
Date: 04:23:56 02/14/03
Go up one level in this thread
On February 14, 2003 at 07:10:40, Rolf Tueschen wrote: >Just to explain some basics for new readers, I show why the whole List is >worthless. The rankings are by chance the way they are presented. > >Since only a few here have basic knowledge in statistics I explain the most >apparet things. > >We are told that for instance the two first programs are seperated by 8 points. >No matter Stefan get all the credits here for his first place. But is true that >Shredder is stronger than Fritz? I do not know but it is better guess to guess that shredder7 is better. I wrote in my response: "This time it seems that sendro was right when he guessed that Shredder7 is better than Fritz7 and I understood that it came to be number 1 inspite of not using the classic interface." In case that I was sure I could avoid the words "it seems". > >Here I must tell you that we simply don't know it. The SSDF pretend to know it, >but it is NOT true. How can I say such things? Easy! Look at the deviations. >These numbers with + or -. We see that most programs have an expected Elo number >varying plus/mius of about 30 points! Note, that the Elo minus 5 is as probable >as the fially given Elo for the ranking! > >If you then take a look at the Elo of the opponents in the far right you can see >that even for the top programs the SSDF was unable to create equal conditions. >Also this influence by different opponents makes the 8 numbers difference at the >top meaningless. I do not think that it is meaningless. I give shredder bigger probability than Fritz to be number 1 based on my knowledge. The probability may be only 55% (I did not try to build a model to calculate it) but it is enough for me to say that shredder7 seems to be better than Deep fritz7. Shredder7 is probably better than Deep Fritz7 in other words. > >In sum we can say that the SSDF failed to show - exactly what they pretend to >show - the differences between the actual top programs. The SSDF presents a new >leader, but that is against its own results! So that the conclusion is allowed >that SSDF makes deliberately their own new number 1! > >(Note please that this is not a political speech, however it is what statistics >demands. The SSDF got this critic so often in the past but they still did't >change their experimental setting.) > >Rolf Tueschen I believe that the ssdf did not choose opponents for shredder7 in order to do it number 1. There is a statistical error and other errors in the rating but we have no way to know what is the direction of the errors. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.