Author: Thorsten Czub
Date: 10:50:54 12/13/97
Go up one level in this thread
Ooops - now I am in big trouble because I have to disagree with Bruce and Don together ! Hm... >How right you are! In testing I've done, I have seen scores more >lopsided >than this based on 100 games or so turn completely around after another >200 games or so. It's to be expected. May be. But scores is not important ! The games tell about the ELO, not the score. The score various from each game, win - loss - loss - win - win etc. ELo depending on score goes up and down. But I don't need to build the ELO from the SCORE ! I can try to estimate the ELO by looking in the games. If you evaluate texts from pupils they have written about their last holidays, you can give remarks about how many mistakes they have done (typing and writing errors = quantifying method) or by evaluating their style and expression and fantasy (qualitfying method). > >I remember a company once years ago producing a glossy document >descibing >and annotating a 10 game match between it's new chess computer and the >previuos model. It was Morphy vs the spraklen 2.5 program if I remember >correctly. The match result was 7 to 3 in favor of the new thing. But >this kind of result is even less reliable than the actual score would >suggest because you have to realize that they would not even be printing >the results had it not gone their way! This advertising probably did >appeal to the general public and it was fun reading (and maybe Morphy >really was stronger) but I definitely took it with a grain of salt! Very OLD memories Don - :-) > >>Some people think that they can tell which of two programs is stronger, >>by eyeball, but I'm mistrustful of this. If one program beats another >>7-3 (a statistically insignificant result), which one do you think they >>will pick? How often do you see computer vs computer losses in which >>the loser looks good? > >I'm extremely distrustful of this too. Another anecdote: Me and some >friends >were playing around with the super constellation and then one of the >Richard lang Mephisto things years agon. They were strong chess >players. >After >lot's of fun speed games, human vs computer, they gravitated toward the >Super constellation, believing it to be much stronger. But I remember >that their results were better against the Constellation. I told them >the Mephisto program was much better (based on many games I had played >between them.) They did not believe me. So we played 3 or 4 games >between the 2 computers and the Mephisto kept winning. They believed >the Mephisto was luckier and didn't like it's style of play! In their >own >games they discounted their losses to the Mephisto (you know, "I just >have won that game") and were impressed by the Sacrafices the >Constellation would often >play. Right. But don't you make a mistake by saying that STRENGTH is absolute ? I remember that John Nunn once was impressed by a computer because it played "very strong". In fact the computer played misable against my friend Bernd Kohlweyer, not a strong IM (only 2420) but enough to smash this machine into targets. WHY was John Nunn impressed ? Because the machine played like himself, tactically. We are always IMPRESSED when somebody holds a mirror of ourselves into our direction. Try it with an ape, and he will be impressed too. but does this say ANYTHING about strength ? No. And your score's ? If you know A wins against B in 75 : 33 games ! And you know A's ELO, do you know B's ELO ? Really ? I don't think so. If you know a girl and she kisses nice, and you know another girl that kisses only 2.3 times weaker, you don't really KNOW this girl ! You don't even know her strength. All you know is how she kisses in relation to girl A. And all you can say after the match A against B is: a plays (against B) xyz ELO stronger. I don't think A will get the same amount of score he got against B, against a bunch of ELO B players. When is a writer better than another writer ? How can we measure this ? Is Hemmingway better than London ? Why ? Is Goethe better than Shakespeare ? How will you ever measure this ? I doubt this. When do we come to the point that we understand that NOTHING in the world is deterministic ? Of course we always try to make it deterministic. We like EASY statements like: A is better B. Because a short way is faster to follow than a long way. Also a short way gives earlier results. But - and here I am sure, I can compete ANY competition with an autoplayer. I will always find out much earlier than any 40/120 autoplayer-system HOW STRONG a chess program is. I don't need 100 games. Nor 50. And if the scientifical approach NEEDS this much games to be precise, than the method they use is senseless because it is beaten by another method that is much faster and as precise (or even more precise). Please, don't get me wrong: ANY guy can measure this, if he can feel it. it is not me. It is the long time I have done this. I am sure any wine-expert is as good in testing wine and does not need machines and experiments, I am also sure any mechanician is much better than any machine in finding out which car is better.... So, please let me survive now... although I am against your opinion. It is christmas time again. > >People are not very good judges of things like this and they often >prefer style to substance. I rarely trust results people report to me. >-- Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.