Author: Carsten Kossendey
Date: 09:17:45 03/17/98
Go up one level in this thread
On March 17, 1998 at 06:25:55, Ossie Weiner wrote:
>***************************************************************************
>The Fritz 5 scandal
>
>International competition manipulated by a Hamburg company?
>
>***************************************************************************
It is quite common to make false claims and disguise them as questions.
The German newspaper "Bild" (among others) is quite commonly known for
this. Your whole posts (and I mean ALL I actually ever came across) are
the same low quality as Bild's articles.
>1. Manipulation - the formula for success?
[snip]
> A typical "formula for success" can be described as follows:
>
> MANIPULATIVE METHODS ---> leading to --->
> UNJUSTIFIED ADVANTAGES ---> leading to --->
> UNDESERVED SUCCESS
>
> * Manipulative methods, e.g. exerting pressure on a
>non-commercial
>organisation like the SSDF in order to change the rules in the favour of
>one specific company.
No rules were changed. The SSDF Fritz5 version demands certain hardware
and software to be present in order to work at a reasonable level. This
is perfectly legal. Nobody would complain if a program refused to play
without an opening book, or without endgame databases, if the absence of
either would clearly hurt performance.
If Crafty were to be tested by the SSDF, Bob Hyatt would most likely
require that (a) the standard set of TBs as well as (b) a reasonably
sized opening book was used and (c) that the book was copied to hard
disk so that book learning can take place. Everything else produces
sub-optimal results and hence is not desirable for the author.
> * Unjustified advantages, e.g. by using of a very special,
>top-secret, publically not available autoplayer system, such receiving
>some
>valuable "gifts": a huge amount of extra RAM, add-on powerbooks and end
>game databases, access to opponent's data, ...
While it is certainly bad that the Fritz5 Autoplayer is not available to
the public, the concept is not quite new. There have been special SSDF
versions of software for quite some time now which were not available to
the public either (such as DOS versions of Windows programs).
Also it is up to any programmer to make assumptions about his opponent
and modify the behaviour of his program accordingly, just like Crafty
notices (when playing on the Internet) that the opponent is a GM/IM/FM
or a computer, and modifies some parameters accordingly ... This is just
the same as a 2000 level human playing more carefully when facing
Kasparov as compared to possibly making (even unsound) sacrifices
against a 1600 level player.
> * Undeserved success, like an inflated rating, which seems - not
>surprising after all this extra privileges - almost 100 Elo points
>higher than in handtested games.
A good deal of Fritz' SSDF games is available to the public. Numerous
people have tried without any success so far to locate moves with the
"out-of-the-box" Fritz will not play in contrary to the autoplaying
Fritz.
Also, the Fritz5 SSDF rating is based upon 200 or so games. Where are
your 200 hand-tested games, for a start?
And to come back to a previous point, when a program plays via the
Autoplayer, it can make assumptions about its opponent. You didn't
eventually give Fritz any hint about its opponents (like their ratings)
in those hand-tested games, no?
> There are only two ways to face such (political?) situations:
> * One can either start using the same methods,
"Start" is evidently not quite correct since everyone is already using
one trick or the other; be it cooked opening books, commercially
unavailable program versions or simply tuning parameters in Autoplayer
mode.
> * all parties together can try to stop this nonsense and return to
> the previous status.
The previous status being only using results achieved via manual
testing?
>2. How the Fritz 5 scandal started
>
>For many years the SSDF rating list has been a trusted source of
>information for computer chess enthusiasts. It used to be the basic
>principle of this non-commercial organisation that chess computers and
>chess programs were tested only in exactly the same form which was also
>available to the public. This straightforward principle made the SSDF
>ratings valuable for all interested parties.
>For the first time now the SSDF has allowed a company called ChessBase
>GmbH to supply a special hardware/software configuration which is NOT
>COMMERCIALLY AVAILABLE.
That was not FOR THE FIRST TIME, no matter how often you repeat it.
Where have you been on previous examples of this? Oh I see, you're
selling that other software yourself; well I would try to ignore the
truth as well then, I guess.
>That means SPECIAL PRIVILEGES have been given to
>ChessBase in comparison to other chess software suppliers.
>
>a. Unlike all other chess programs Fritz 5 has been tested with a
> special Powerbook loaded onto the hard disk for speed increase.
I don't really think copying an opening book to hard disk gives you ANY
speed advantage compared to having it on CD-ROM if the book is designed
for random accesses. The key difference is, however, that you can write
data into the book if it resides on hard disk so that learning can take
place. If a program does not learn from its mistakes it is worthless
anyhow, since even the weakest player will be able to beat it - that is
only a question of trying often enough.
Also other programs load their opening books into memory, why don't you
complain about these?
>b. Fritz 5 has been tested with endgame databases. This has not been
> the case with other programs such as MChess Pro 7.1 and Shredder 2.0.
It is up to the authors of these programs to require that the data they
supply is actually used.
>c. The exe-file used in these tests is different from the commercial
> Fritz 5 exe-file. For example unlike the commercial Fritz 5 it's
>refusing to play with less than 64 MB RAM or without special
>opening books.
Since the commercial Fritz5 does not have autoplayer support it is quite
clear that the executable is different, no?
>d. Fritz 5 is not being tested with a standard auto232 interface, but
> with a very special SECRET CONFIGURATION supplied by ChessBase.
Umm ... when since is there a standard auto232 interface which supports
Fritz5? You don't have a point here.
>e. This setup opens the door to various manipulations such a special
> tuning to different opponents. It also requires a min. RAM size of 64
>MB, a UNIQUE PRIVILEGE nobody else has demanded or been granted up to know.
Using the Autoplayer alone is a sufficient criteria for making
assumptions about your opponent and tune for that specific group. I
assume everybody is doing so.
>g. Independent experts have calculated only the HANDTESTED games of
> Fritz 5 where the achieved rating is VERY MUCH LOWER. Evidence for
>manipulations?
No. Proof of statistical laws. Less games provide less accurate results.
[snip]
>4. Manipulation by autoplayers
[snip]
>Now there is even EVIDENCE that the CB-autoplayer has access to
>opponents'
>computers and can even overrule certain files (e.g. saving games doesn't
>work anymore). Can significant influences (book learning, auto-learning,
>time controls, ...) be 100% excluded?
This is the biggest bullshit I have ever heard, sorry.
You say I plug a device into a port and the computer on the other end
starts controlling my machine? This is NOT gonna happen unless you
explicitly write your program so that it can happen.
>Why shouldn't ChessBase be using the most sophisticated methods they are
>able to? Why not, as long as it's technically possible and being
>tolerated by some friendly people ...?
Programming a computer chess engine is ALL about using the most
sophisticated methods one can. However you seem to have no idea what is
technically possible and what not.
>5. Powerbooks and killer books
>
>Everybody knows that ChessBase has plenty of autoplayers and has
>probably
>thousands of test games against all competitors on record. So nobody can
>believe that this research may not have been used to compile their
>Powerbook.
Now since the Powerbook ("Powerbook" was actually a trade mark of Apple
Computer last time I checked, but this is getting offtopic) is
commercially available, what in the world is holding the other
programmers from trying out all the lines against their programs and
modify their books accordingly so that their programs don't play lost
lines?
[snip]
>What does that show? With super-dooper Powerbooks one can statistically
>achieve phantastic results against particular opponents, but at the same
>time totally fail against others.
With ANY opening book you will play seemingly brilliant against against
one opponent and fail against others. Should a GM dare to play the Two
Knights' Defense against me with black I'd be more likely to win than
in, say, a Sicilian Dragon. And this is not because the Two Knights'
Defense is inferior to the Dragon but because I can simply handle the
Two Knights' better.
This is not quite new. Every human as well as any program doesn't play
all openings equally well. It is up to everyone to try and lead the game
into lines where he/it feels comfortable. This is highly desirable, not
a form of manipulation.
>6. New evidence for unbalanced competition
[snip]
>How is Fritz 5 doing in MANUALLY played games? Does it make a
>difference?
>
>THE ANSWER IS YES! THERE IS A BIG DIFFERENCE!
This is getting boring, you're repeating yourself over and over. OF
COURSE IT IS AN ADVANTAGE TO KNOW WHO THE OPPONENT IS <sigh>. If the
Autoplayer is in use that limits the group of potential opponents down
to a VERY small group. Every programmer can take advantage of this
knowledge, now don't complain if someone is actually clever enough to do
it while the programmers whose programs you sell are not.
[snip]
>Conclusion:
>It can't be denied that there is a difference of ca. 80 - 100 Elo points
>(!!) between autoplaying and manual results. How fortunate for our
>Hamburg friends, that the SSDF is using the special ChessBase autoplayer ...
The same will happen if Crafty plays on a chess server and knows that
its opponent is a computer, as compared to letting it think it is facing
a human of whatever the default opponents' rating is. No big deal. No
deal at all actually.
>7. ChessGenius 4 - a cheap excuse for ChessBase?
[snip]
>NOTHING OF THIS IS TRUE !!
>
>a. ChessGenius 4 (DOS) has been available since 1997 inside the
>ChessGenius Gold Collection.
1997 isn't 1995. There's a two years' difference. Now if ChessBase
promises that it will publish the "secret autoplayer" in 2000 will you
finally give peace?
[snip]
>d. On request of the SSDF Richard sent a DOS version to Sweden with
>identical engine and identical opening book like the Windows version.
>The SSDF had the opportunity to compare both versions and make their own
>decision.
Unless you can PROVE that the SSDF Fritz has an engine different from
the standard one I wouldn't recommend claiming that.
Also it was up to the SSDF to not use the Fritz5 Autoplayer version but
conduct manual tests using the out-of-the-box version.
>g. One year lated Richard went from floppy disks to CD-ROMs and
>published
>both the Windows and the DOS versions of ChessGenius 5 together. This
>worked strongly AGAINST him because it enabled all competitors to play
>autoplayer games against ChessGenius 5 and tune their programs
>accordingly.
Now even if you HAD the Fritz5 Autoplayer right now, that wouldn't help
you to manipulate the ratings of the programs already on the list, would
it? Whoever gets the most time to prepare himself achieves the best
results, there's nothing mysterious about that...
[snip]
>Richard just wanted to be polite and help the SSDF, not to gain any
>advantage. Richard did not use manipulative methods - that's the big
>difference!
By not making the Autoplayer version available, Richard gained an
advantage whether he intended it or not.
>8. The SSDF - an organisation in trouble?
[snip]
>If an autoplayer isn't commercially available it should be simply banned
>from testing. Otherwise the SSDF rating list will soon become worthless,
>as "out-of-the-box" programs won't be rated anymore.
The SSDF list is worthless since they started playing programs across
various machine types. I don't give ANYTHING on ratings achieved by
playing a P200MMX against a P90. If a program is to be rated it should
ONLY play against other programs using the same machine speed. If a new
platform is introduced the ratings for the existing programs on that new
platform should be produced by ONLY playing the SAME version of the SAME
programs on an existing platform (but not adjusting the ratings of the
programs rated on the old, slower platform)
>9. The elegant solution: The UNIVERSAL AUTOPLAYER!
[mega-snip]
>If it can be shown that Fritz 5 is the top chess program with REGULAR
>methods (without special autoplayer, without special Powerbooks, without
>more RAM than the competition), then I will be the first one to
>congratulate.
Noone holds anyone else from giving Fritz' opponents as much or more RAM
than Fritz gets, I assume?
[snip]
> * As soon as the facts are on the table, the scandal will
> be history.
There is no scandal. It's only you and a few others not wanting to
accept that Fritz5 is the strongest program currently available.
> * As simple as that.
As simple as that.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.