Author: Rolf Tueschen
Date: 15:22:44 12/04/02
Go up one level in this thread
On December 03, 2002 at 15:49:45, Robert Hyatt wrote: >On December 03, 2002 at 12:54:24, Rolf Tueschen wrote: > >>Until now nobody out of the programmer group had ever spoken about that evident >>truth. SMK says that these tests can't show the strength of play or as it was >>claimed for this test, the "ability to analyse". SMK also explained (for the >>first time in that direct speech) how he and every programmer could fake the >>results of such tests. He then speaks about the question if it could be >>discovered, as it was by T. Mally in case of Ed Schröder, and he saud that of >>course he could do it so that nobody could find out. In fact he had written such >>a "tool", but in the end he decided to let it out of the commercial product. > >This is a half-truth. I think we have a language problem here. I quoted Meyer-Kahlen who had said that, just shortly before I wrote my message. I did not say that Ed said this or Mally. Sorry, that my poor English caused that one. [Buhut, your lines are still toooo long. :)] > >Test sets can and do measure tactical skills. But that is only a part of the >game. You >also have to play moves when there are no tactics to deal with, and that part of >the game >is uniquely unsuitable for most test sets. And that was exactly what I was trying to say. Physically impossible sure is a bit exaggerated for the effect. > However, too many rely on test sets >as an >"absolute truth" and that leads to flawed or completely wrong conclusions... Dr. Michail Gurevich, I always thought the GM, but he denied that on CSS, but he said he played in the German Second Legue, well, he promoted his test over the whole year now (the actually new tasks were always published in the German journal of Friedel et al. So it was also a bit a commercial project so to speak.)with phrases like "I think this test is the best and it is my life-work" and stuff like that. Like a religious project more or less. He proudly presented his claim that he had replayed all Wch games etc. But to me as a layman it would be much more difficult to construct test positions wwhich then would really / possible test certain aspects. But he took mainly the hottest positions from literature (with expert commentaries already made). Exactly against this strategy I wrote my message here. And you made it even clearer. What is with the development of the game up to that position? Perhaps the machines are still blind for these preparing moves, that are so hard to find. For weaker players too. But then you take another aspect under your examination: > >That said, I would disagree with the idea that there is no way to test >positional knowledge >with a test position. That is simply wrong. IE if you want to see if a program >understands >(a) having a distant pawn majority is good and (b) if you have one, the less >material left on the >board the stronger it becomes, then it is possible to create a position that >will reveal that the >program clearly does _not_ understand the idea, although it is nearly impossible >to create a >position that shows that it _does_ understand the issue. But this is my _point_! If we could make a position where the silent moves led to the crucial positions, then we could tes if the programs would go for it. But taking the crucial position that method is surely unable to show if the program had the "understanding". I had this debate now for two years and I had difficulties to only make my point clear to the others. And I was not inhibited by the English then. People, also experts, are so blinded by the practice of testing, that they can't see what I am talking about. Interesting that for you this seemed to be no problem. So, we discuss now the question if we could say that there existed positions that could test etc. But then you admit yourself that we can't be sure that the progs play the key moves for the right reasons. Well, that is what I would call not the right position for a test (positional play). But ok, the truth may be lying in between. But taking simply positions from Wch and then calling the test Wch test - - that is a bit naive. >IE programs are pretty >good at >making the right move for the wrong reasons. But it is very difficult to make >the wrong >move for the right reasons since that is a contradiction. :) I think I tried to present such a position in one of my Misc in rgcc IMO. I think this would be not impossible in chess. The program understands exactly, makes the move, but because of horizon effect it is still not winning or drawing... However - - If you define "understanding" in an absolute sense, then I am just dogging me around, but since when chess is understood in that absolute sense? We have no absolute truths until chess is solved. Agreed? > > >> >>But all this gives me the opportunity to talk about the reasons why such a >>testing with even these top class positions is nonsense. And why it has nothing >>to do, well, almost nothing, with _real_ strength. >> >>I think I can show you why in special for those allegedly positional positions >>the test is nonsense and that he's measuring something else, but not analysing >>power of the engine. >> > >It is mainly measuring (a) the tactical ability of the engine; (b) the >aggressiveness >of the engine's evaluation. Because most test positions involve something to do >with >winning material, or exposing the king to a violent attack, or producing some >easy to >see endgame winning idea... Right. But strength would be defined for finding the moves prior to all these positions. Remember the saying of Tartakower who said I wished to get these positions Alekhine always has on the board, but no matter how hard I try I can't get them... :) [He meant that if he had the positions he could also win the games! This is not understood by amateur players who always pray that this time they get such positions. They simply don't know that you must prepare for. That can't succeed with some wild idea and then full praying mode. I was always good in playing fantastic moves, but alas, I had no knowledge of the necessary context. I always hoped that by magic my move would do it all by itself. Just exaggerating a bit.] Rolf Tueschen > > >>I will keep it very short so that you can do your own research. >> >>(Just to mention that I asked for that problem already two years ago as >>'Schachfan' in CSS forum, but then it went about a tactical mate position). >> >>Look, if you have a positional game of chess, where do you choose the point for >>a test? Of course, in this WM-Test of Gurevich et al you take the position when >>exactly a certain by the experts well commented and mostly beautiful move has >>been made. Because there the commentators said: only with this move he could >>conservate the slight advantage.But the truth is that often the engines see - in >>their actually possible realm - two solutions very closely together. And in >>general it could be said that for positional positions without tactics the evals >>are not very impressive at all. So, how could you calculate it in your results? >>Would you really take a difference of 0.01 points as decisive? Is that relevant? >> >>But the main problem of such test positions is this. >> >>The point of that "nice move" (that caught th attention of the commentators) is >>by no means the most important moment for the decision making. Let me explain >>the irony. The usual commentators are masters themselves. Well, and therefore >>they take certain decisions as completely normal, because they are easy and >>trivial for _them_, but not so for the amateurs. Or the machines so to speak. >>But now go with me bachwards a few moves. How optimistic you are that we could >>then expect that a machine would be better prepared to make the right decision >>in such _positional_ games? And that is exactly the point for these test >>positions. _Realistically_ we had to test the machines in positions, where only >>experienced humans know how to play to be later in the position to make some >>"decisive" moves, moves then commented by our experts. Only the early positions >>would allow a verdict if our actual machines could play posiional chess. We know >>already the answer. They can't for the moment. >> >>But therefore such tests with such a great pretension are a fake, a hoax in >>themselves. And Stefan MK explained it with the possible distinction. In reality >>M. Gurevich is making a question of life or death out of it. But earlier >>somewhere I already mentioned that it's ridiculous to claim the honor for a so >>called, guess that, I translate, World Champion Test. These positions are simply >>taken from Wch matches. What a thrill! But it's known for ages that the chess of >>these matches is not always the best possible. Because it's mainly a >>psychological fight. And fortunately Gurevich didn't claim that he were testing >>psychology. But just now it was published that one position wasn't from Wch >>chess at all. A game between Anand and Shirov. And to make the scandal even >>greater. The authors used a false position. Instead the K stood on c7, they put >>him on d7. But with Kc7 we have two solutions. The searched Ng5 and now the odd >>Bg5 too. Christ! A whole life work of a few hours of choosing some positions out >>of Wch games is in danger to lose all reputation. Doctor doctor, gimmi the >>news...! >> >>Rolf Tueschen >> > > >Anyone that believes ratings produced by test sets is simply naive. Anyone that >believes >that a test set can prove that a program is playing at a world-champion level is >simply >stupid. :) > > > > > >>On December 03, 2002 at 09:26:42, Eduard Nemeth wrote: >> >>>Very interesting post from SMK in CSS Forum (only german). >>> >>>Please read it, i thing that a translation is interesting for You! >>> >>>Read here: >>> >>>http://f23.parsimony.net/forum50826/messages/54995.htm
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.