Author: Sandro Necchi
Date: 01:43:45 03/06/04
Go up one level in this thread
On March 06, 2004 at 04:28:25, Uri Blass wrote: >On March 06, 2004 at 04:17:14, Sandro Necchi wrote: > >>On March 06, 2004 at 03:25:50, Thorsten Czub wrote: >> >>>On March 06, 2004 at 03:10:22, Sandro Necchi wrote: >>> >>>>They do their best to give reliable data without favoring any program and >>>>checking the games before recording the score. >>>>This is how they always did. >>> >>>this is what we are discussing Sandro, if they give their best. this is part of >>>the critics for years now. because people have different kind of standards >>>concerning this "they do their best". >> >>I agree on this, but they cannot find something that everybody will agree on. >>I was stating that they test the commercial programs with the strongest >>settings. To do that they ask the programmer to advise the strongest settings. >> >>I said that I did not always agreed with them, so I am not saying that I think >>how they do it is the best, but I know they have been trying to find a way to >>get the most reliable data. >> >>Of course suggestions are welcome. When I gave them some, most of the time they >>were accepted and everytime they lesson and talked about them. >> >>Since they are independent, we cannot force them to accept our suggestions, but >>if you give them good suggestions I am sure they will consider them. >> >>> >>>IMO it would be within a normal range of "doing the best" to make sure that a >>>company is not getting special advantages due to a special autoplayer system. >>>also IMO it is within the normal range of "doing the best" to make sure >>>double-games are not counted again and again, therefore you need to collect the >>>data and make them public so that all testers can see when they play double >>>games. We have seen this list beeing presented like a holy grail for many years. >>> >>>and in the years of the dedicated chess computers the double-game topic was IMO >>>very relevant due to the fact that the books were small and the variety of >>>openings and games was very limited. >> >>Yes, but on this matter I do not agree with you. If a program has no learning >>feature and will play a lost game more than once it has to be tested as it is >>and the double game included in the list. >>Why? >>Because the owner will be facing the same situation in a match against it and >>since this effects the streght of the program it must be included to get more >>realistic data. > >There is a second possibility >Not to include programs that behave like that in the list. > >Suppose that program A has no learning and can lose the same game again and >again. > >Program B played against A 40 games when program C played against A only 15 >games >Is it fair that B will get better rating than C? > >Uri Uri, no, I do not agree with you. The SSDF is requested to test the commercial programs available. If they do not they may be criticized by some people (like with CM9000). Since they test ONLY the programs that are requested to be tested. The distributor sends 2-3 copies to them for testing, they do what they are requested. Your request is like...do not include my tournament games as I was sick and therefore did not play as I should/would. The way SSDF do is to test in matches of 40 games between 2 programs. It may happen that when the list is pubblished not all the match games have been performed, but I am sure they do not favor any program in a way or another. If I would state something different I would be dishonest, which I am not. If a program is without opening book should be tested without opening book. If it has no learning features it should be tested without. I MEAN AS IT IS!! IF WE START FAVORING SOMEONE IN A WAY OR ANOTHER WHICH IS THE LIMIT? WHY THEM WE DO NOT REMOVE BAD GAMES, WRONG OPENING LINES GAMES, TACTICAL BLUNDERS AND SO ON...and so on. Regards Sandro
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.