Author: Moritz Berger
Date: 10:06:18 04/28/98
Go up one level in this thread
On April 28, 1998 at 04:00:31, Ed Schröder wrote: > Then I hear about the use of power books, endgame Cdroms which are > not part of the commercial package of the program. Minor points IMO > concerning the gain of some extra elo points. But another proof of > the tendency that you sin against your own rules. The endgame databases (only a couple of the 5 man ones) are included on the standard Fritz 5 CD. They are part of the commercial package, even if Ossi Weiner of whoever might claim otherwise. Please verify at least the facts before giving us new rumors. > >#4. But my main criticism is that you have accepted an unknown piece of > software for autoplaying on your list. The autoplayer behaves >totally > different then the general accepted auto232 software. And then the > worst came, the new autoplayer is secret. Again you sin against > your own rules. Alternative would have been: Manual testing for the SSDF and autoplayer only for internal testing at ChessBase. What would you have gained? ** NOTHING **. So your true demand should be spelled out clearly: Don't accept any program on the SSDF list which doesn't have a publically available Auto232 autoplaying facility. That would exlude of course e.g. ChessMaster 5500, Virtual Chess and others. Also the TASC R30 ought to be taken off the list, naturally. >#5. You know very well that auto232 is a fragile piece of software and > easily can be miss-used still you accepted that other competitors > or other participants on your list were simply sent in the dessert > as nobody could check this piece of software. Verification by independent technically savvy experts ought to be enough (e.g. like your cooperation with Enrique). If there's serious doubt about a "cheating" autoplayer, just use a 3rd PC as RS232 protocol analyzer (see Dr. Dobbs Journal some 1997 issue for the complete source code of such a program that would be of course capable of handling the 1200 BPS auto232 protocol ...). Many programs have logfile options, making cheating virtually impossible even without resorting to such massive efforts like protocol analyzers etc. >#6. As a result of that (and this is not funny at all) I made a special > Rebel auto232 version which checks every auto232 opponent for fair > play. This version was sent to somebody who has the secret Fritz5 > autoplayer. This special Rebel version reported, all ok. Yeah, that's the way to do it. Facts rule. Well done, Ed. >Now for the future of your list I recommend you the following >improvements >which is a simple request to return to your basics and your own >rules.... > >#1. Only test programs as they come in the commercial package. What about including a "SSDF" option in your program (like Hiarcs kind of does by displaying an exclamation mark in the menu bar item "Hiarcs!" if everything is set to maximum strength)? This way you would also avoid book option hickups etc. >#2. Also test freeware programs like Comet, Crafty, Decade as they are > distributed. All these programs have in common that they are > generally available to the public. Nothing should be hidden. Why don't demand that the sourcecode by publically available? Crafty certainly complains with this demand and might well end up as new #1 if we make this a must have ;-) Just kidding ... >#3. Do not accept any add-on's like separate books, endgame cdroms if > they don't come with the product. DISAGREED. Just a marketing decision of ChessBase: Fritz 5 "standard" edition comes without PowerBooks, Fritz 5 "professional" is "standard" plus "PowerBooks". Just a matter of spending the money on the "pro" version, isn't it? As long as the SSDF makes clear which version gets tested, everything would be alright with me. >#4. Do not accept autoplay software that is not public available. Stick > to the standard auto232 protocol all programmers have agreed on. DISAGREED. If it can be verified that the driver is clean (see above), I don't see a point in having the SSDF playing out games manually. >#5. Test programs on equal hardware. Do not accept special demands which > gives a program an advantage above others. DISAGREED. Test on equal hardware, yes. Accept special demands, also yes, but then give the other program the same hardware (e.g. when both get rated as P200MMX both ought to get the same amount of RAM). When demands are unreasonable (e.g. 128 MB RAM might have been, although Fritz would have gained something from this demand), reject entry. Better yet: Fully standardize hardware (CPU, RAM), see below. Fritz needs opening book on harddisk for book learning, special demand for the SSDF, of course, but unreasonable? I don't think so. >#6. Test with equal hash table sizes as far as this is possible. Do not > accept demands for a minimum hash table size other then the system > requirement that comes on the box of the program. Everything else is > a clear try to get an unfair advantage which you should not accept. SSDF tested Fritz 4 with 512 KB hash tables, without permanent brain, running in multiple variant mode. This was not forbidden on the box but shows that a clearly defined standard configuration (ensured by the program itself when running in "SSDF test mode") is beneficial for the accuracy of ratings. Why don't you (commercial chess programmers and SSDF) agree on 64 MB RAM as standard on P200MMX hardware for all programs? Doesn't sound unreasonable to me. >If you return to these rules again which were common in the past it will >be my pleasure to compete on your list again. No, the rules in the past were flawed. We need better ones. Fritz 4 was rated 70-100 points too low because of ridiculously insufficient hash tables, wrong configuration etc. Hash table size is a very important factor for some programs, so the list needs at least to be improved in the respect that Fritz gets rated as "Fritz 5 / P200MMX / 44MB+ hash tables" (and likewise all other programs). My demands would be: CPU+RAM equal for all programs (in each CPU class, of course). Accept (publically available) "SSDF mode" for maximum playing strength in all commercial entries (to make sure that book settings, permanent brain etc. is in order to maximize playing strength). To make the Auto232 interface publically available is YOUR DECISION and a major feature with some customers, i.e. I will buy Rebel because it supports my Chess232 board and will not buy Fritz because it doesn't. It's an important feature that Fritz doesn't have, so you sell more copies of Rebel, right? That's why I think that ChessBase's decision might not have been as wise as they probably thought it to be ... Moritz
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.