Author: Rolf Tueschen
Date: 13:54:59 06/13/04
Go up one level in this thread
On June 13, 2004 at 11:03:11, Robert Hyatt wrote:
>On June 13, 2004 at 09:39:07, Rolf Tueschen wrote:
>
>>To Mike Scheidl, Christian Liebert and other CSS believers in the WM-Test!
>>
>>
>>
>>You know now that after the verdict of Bob Hyatt your CSS WM-Test is dead. All
>>what you write now is typical for the censorship in CSS. You want to define what
>>something "in reality" "means" or better should mean in the eyes of CSS.
>
>That goes _way_ too far. I simply said that in my opinion, the concept of using
>test positions to predict ratings will not work, has never worked, and is not
>worth losing sleep over...
What I wrote was related to the concrete reality of that CSS Test. What is its
reality? That people like M. Meiler are proud to present the Excel results of
over 300 engines in the meantime. These results are presented as the best
estimation of [no, NOT the strength!] the "ability to analyse" - the result is
given with an Elo like number. Now if I take what you are saying, this is
impossible to do. Simple as that. That intention is dead. Period.
The fact that such positions have a meaning on their own - who would doubt it?
The positions are almost all taken from games of Wchamps. But
Gurevich&Meiler&Schuhmacher are not interested in debugging. They are chess
players and testers. They wanted to know the "exact" "number" for the
"analytical abilities" of a chess program. They publish rankings and so forth.
All that is normally - if following your convincing statement - now a nonsense.
That intention is dead. Period.
That you are modest enough to declare all this as your personal "opinion" is a
good thing because the censoring artillery in CSS did already insinuate the
opposite into you as if you had fun in playing the Highest Judge with your
"verdict". <grin>
>
>
>>
>>That Stellungstests, position tests, show flaws all over, from the start, from
>>the ide, from the calculations etc pp, that is clear by now.
>>
>>Now you sing that song of the hooby testers who want to have just --- fun. If
>>that were the case then why the "Weltmeister-Test" is presented by CSS as the
>>first in the chapter about The Best Test...?? Is it allowed to conclude that you
>>thought it were the best test? Of course because you didn't just list them up
>>alphabetically. No. Weltmeistertest comes almost at the end of the alphabet but
>>that Gurevich test still is on number one place. Why?
>>
>>Because it's the test that brings the people the most fun??? No!! Of course not,
>>but because it's the best test suite, following the makers of CSS.
>
>
>One reasons test sets are bad is that programmers will tune for them, to better
>their results, even if it means that the engine ends up slightly weaker. Once a
>set of positions is published, results can't be depended on.
Your "opinion" is becoming more and more embarrassing for CSS, because that
German journal wanted to present that test as the best available on this Earth.
They simply leave you alone with your story about IM Larry Kaufman. Schuhmacher
& Gurevich are both decent players in the 2nd Bundesliga in Germany!! They can
analyse chess positions almost as good as masters or GM. Therefore the highest
reputation for the CSS test... And that also thanks to the good PR of the
journal.
>
>Test positions are useful for debugging. They are not so useful for trying to
>predict a rating. And spending time trying to do so is just a waste...
Bob, perhaps this is news for you but they dont calculate ratings just for odd
strength!! They calculate ratings for "analytical abilities" of the machines.
Could you comment that? Please.
>
>
>
>
>
>>
>>
>>Hey you guys, let's please relax a bit!
>>
>>We ALL know here in CCC, that you, personally, are not that stupid to believe in
>>Stellungstests and its ratings. Of course you want to make fun for your
>>readership. That is clear and well understood.
>>
>>Now what you should do in the future?
>>
>>Just this:
>>
>>If someone - mostly people from universities or other statistical students -
>>writes some questions about the WM-Test, do by all means clarify, that Michael
>>Gurevich, academic doctor BTW, has no chance to answer himself. Because he's in
>>a terrible double bind as test author. He has the typical black spot as an
>>expert. He did all correctly in his test suite but he did NOT reflect what the
>>100 World Champion Positions could maximally bring to the users. You say, and we
>>all agree, it can bring much fun. Yes, that's it. It's fun to follow the
>>analyses of the machines in such valuable human chess positions.
>>
>>BUT - a big BUT!
>>
>>Theoretically and practically that test with ONLY [!!] 100 positions can't hold
>>the water he's pretending he could. He's absolutely worthless - speaking in
>>terms of stats. If Michael G. could collect thousands of positions, only then it
>>would make sense. But of course this is practically impossible to do.
>
>
>
>There are something like 10^50 (2^160+) positions in chess. If you think about
>it, would it not be possible to pick 100 positions where a program could do
>well, even though in inother 2^100 positions the program does horribly? How
>accurate would a rating prediction be, based on the 100 good positions vs the
>2^100 bad positions?
I understand that you doubt the high chess status of our dear test constructors
in Germany? Or do you criticise their hope for basical reasons coming from
logic? That of course could be the nail into their coffin that was still left
open until now. But then they wouldn't be _just_ dead (test builders!) but ready
to be buried. A cold air runs down my spine. Bob, you can be so cruel! Why must
science always be so inhum, unfair and determined????????
You must also consider that these guys have wife and kids. Do you want them to
be starving after their men's death as test authors?
>
>There have been such test positions and timing formulas since the 1980's. And
>each and every one has exhibited horrible "holes". IE IM Larry Kaufman's test
>worked great on micros on particular hardware, because they were all fairly
>equal tactically and his formula and positions were more tactical than
>positional (there were positional tests but they were more than offset by
>tactical tests). But when presented with a program or hardware (or both in my
>case) significantly different from the calibration group of microcomputer
>programs, the formula simply didn't work.
>
>Someone could ask him about the Indianapolis "experiment" where we first tested
>Cray Blitz which suggested it was 600+ rating points better than the best micro,
>then we played some blitz games at horrible time odds and CB won all the games.
>So the test did fine in saying "CB is _way_ stronger than a microcomputer". But
>the formula did horrible at predicting its rating...
>
>Chances are good that with a given test set, assuming programmers have not had
>time to "tune" to it, that the better a program does, the stronger it is. But
>that's as far as it goes. Forget about trying to establish an actual rating
>estimate...
Bob, again, there is no need to be so cruel, unfair and inhuman. Have you no
heart-blood for such dedicated people in our field??
>
>
>
>
>
>>
>> ~~~~~~~~~~~~~~~~~~~~
>>
>>Out of responsibility for my German collegues in the field I want to make the
>>following proposal! [copyright proposal Rolf T June 13, 2004]
>>
>> ~~~~~~~~~~~~~~~~~~~
>>
>>Michael Gurevich and his good team gets the offer from CSS to construct a new
>>test with say 30, 50 or 100 games from the best human chess, games that are
>>suiting the needs of a computer chess tester. Games that contain several top
>>positions - and the test can discover how the emgines behave in context of a
>>real game. Where are typical difficulties for machines... With proposals for the
>>programmers! Etc.
>
>
>Even that won't work. Suppose a program becomes _very_ good at playing closed
>positions. Then the author is going to tune the book to reach such positions
>way more often than what is seen in human games. Using human games to extract
>test positions will therefore extract the wrong kind of positions to predict the
>performance for _that_ program.
>
>Computers have a "style" of play. If you use positions that fit their "style"
>you will get one idea of strength. If you use positions that do not fit their
>"style" you will get another impression. And a wrong one to boot...
Dammit! You do even destroy my own Nobel Prize worthy reflections?? Do you want
to end in hell? You are a real spoilsport!!! Yes, you are.
++++++++++++++++++++++++++++++++++++++
Hi, Bob, I hope you had a good laugh while reading my satirical nonsense. I am
happy that here in CCC we can speak out the truth while over the ocean in
Germany people must follow what the spin doctors of CSS are saying. ;(
>
>
>>
>>I see a tremendously interesting life-long task for Michael Gurevich and his
>>team. They would present the first valid and public computerchess test in
>>history!
>>
>>Please let's look into the future and learn from the errors of our past. Let CSS
>>forum and journal become a place for all kind of creative people, practically
>>AND also theoretically with the combined criticism of the methodology. Then we
>>would all have much fun again with our hobby field.
>>
>>Let's bury the axes of war from the past and let's care for the conditions of
>>the future - because we all know, that the persons who will be joining us in
>>that hobby will increase in numbers due to a complete change in our labour &
>>economical world. CSS and its makers have a guiding role in that important
>>social field of our Nations. But be prepared to use that role in a democratic
>>way without censorship and stigmatizing scapegoats.
>>
>>[Already as 'Schachfan' I wrote such ideas to your CSS chief Steinwender. But as
>>it is in life, often people in power want to have themselves the best ideas and
>>reject the advice from abroad. Often Doctor titles can disturb your vision and
>>anonymous senders are a real pain in special if thought as females. It's not so
>>easy to adapt to new ideas. Therefore a good advisor like me does not ask for
>>recompensations for himself, he can wait until the power people suddenly decide
>>into the thought directions completely on their own! So I do NOT ask for my
>>re-inthronization into the CSS forum. No, my ideas have a better climate to grow
>>up here in CCC, in the international environment. Leave me out of CSS. But - DO
>>something for the best of your German clients! And computerchess as a whole.
>>Pleeeease!]
>>
>>Best regards,
>>Schachfan Rolf
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.