Author: Peter Fendrich
Date: 09:41:38 06/21/99
Go up one level in this thread
On June 21, 1999 at 09:29:53, Robert Hyatt wrote: >On June 21, 1999 at 04:18:57, Jouni Uski wrote: Bob, I mainly agree with you but I think that you give a "one-eyed" view in your reasons. Because programs and humans have completely different abilities they do completely different kind of errors. I still agree with you that the GM's knowledge outperforms the programs tactical abilities, opening lib's, EGTB's etc, but you don't give the whole picture. Comments below. > >>After this 2 - 2 (Cilkchess removed) against super GMs with average ELO >>2621 I think there is no doubt, that PC programs are playing at GM level! >>Actually hardly no further games are needed. And remember, that average GM has >>only 2500 as rating. > >That is incorrect. 2500 is the _minimum_ rating to earn the GM title, the >average is well above this. > > > > >> >>I was at the playing hall in Paderborn. The conditions were excellent and >>sure GMs were playing very serious. I was sitting quite near Fritz - Sokolov >>board and after 14. Bh6 Sokolov's look was "interesting". He starts to think >>about 20 minutes. In the continuation I didn't heard him to resign, but after >>move 24. he simply starts to discuss with Friz operator Feist.... > > > >Let me give you my reasons why I _still_ consider computers to be 2400 >players: > >1. Pick any of the 5 programs that played the GM players. I will find a >game where they played so badly that if you look at _that_ game no one would >consider that program to be a GM. For example, take the winner and look at >the playoff game. Three different GM players commented that they had _never_ >seen white screw up the opening so badly... True, but take any GM tournament and anlyze all the games with any top program. They will find a lot of blunders made by the GM's both in the opening and in the rest of the games. Different abilitites gives different kind of errors. Of cource GM's find those errors horrible... :) > >2. Fritz won in the same way that Deep Blue won the last game vs Kasparov, >actually even worse... The GM simply made a blunder that a 2000 rated player >would spot in an instant. If you take that win away, the even result goes >away. With the same reasoning, this is the type of errors human but never programs does. Even if it's seldom for GM's it happens. > >3. I've been working on chess programming for a long time. And regardless of >how they 'seem' to play in many games, I still know just what they can and can't >do. And they are nowhere near a GM's level in 'knowledge'. They are still >surviving on tactics. And there are plenty of GM players that know how to >squelch tactics and make the game hinge on positional play. And there the >programs simply don't measure up. Not only surviving tactically... > >4. I still consider computers to be 2400+. What is the probability that in >5 games, 2400 players could play evenly with a group of GMs rated 2600? some- >thing like 25%? It should be 20% I think. > >5. GM players exhibit a consistency in quality that computers don't. A >computer will play like a GM for 5 games, and like a beginner for 1. What >happens when the GM players learn what the computer can't do and then >exploit that game after game?. We had at least 4 different GM players watching >part of the games on ICC (particularly the GM games the last day) and it was >quite a common question "how can the machine play this opening, and then play >_that_ move?" Or "why does the machine play so nicely for several games and >then play like a beginner here?" I think this point about GM's (and others) learning how to play against certain programs is the hardest task to deal with in order to reach GM strength for programs. The programs inability to adapt to new situations make them very weak when their weakness is eventually revealed. Opening learning isn't enough here. > >So they are very strong, yes. And if you want to lump these games together >with what we have so far, I don't have a problem with that. So far, I am >counting: > >Humans 5 (counting Anand and Rohde vs Rebel) >progs 3 > >at 40/2hr. That is within reason for a 2400-2600 rating split, particularly >when the just-finished 5 games were played with little preparation by the GM >players, since they didn't know who they were playing 24 hours before they >played. > >Yes, I think the machines are doing well. No I don't think they are 2500+ >_yet_. And the gaps in the program's knowledge are big enough that I don't >think that tactics alone will make up the difference. It goes a long way, >but it doesn't close all the holes. > >One example (I won't name the GM because he asked me not to, and I won't name >the program because I am not certain of the hardware nor that the program was >that which was claimed since I only watched): > >The other nite on ICC, a GM friend of mine asked me to watch 2 games a friend >of his was playing against a computer. The computer hardware was supposedly a >PIII/500 xeon (1 processor) running a program that everyone here thinks is a >very 'smart' one based on NPS. In the two games I watched, the GM made it look >easy. This 'smart' program seemed to not be real aware of the dangers of a >'distant passed pawn' and allowed the GM to create one, and trade material, >until it would see it was lost. I wasn't playing the game, obviously, so I >can't say how 'hard' the GM was 'thinking', but my 'friend' (not the one that >was playing) commented on this. That is a _fatal_ mistake were this program >trying to make its way up the GM ranks, because word spreads, and it simply >gets rolled game after game. I am pretty sure that until a GM finds this >'hole', that program will give them fits on decent hardware. And I am also >sure that once a GM finds that hole, it is over except for the occasional >tactical mistake he will make that the program won't overlook. > >The thing we are 'missing' in the programs is the 'consistency' that GM players >have. IE Shredder played very well throughout the tournament, yet reached a >position in the playoff that I doubt even a 2000 player would reach. almost >zero mobility. Cramped. Inactive. A position unlike one I have ever seen >while watching hundreds of GM games on the chess servers... The problem is >_not_ impossible to solve. But it is _not_ solved just yet. > >This match was 'data'. Let's see how it looks when our data reaches 10 and >then 20 games. So far we only have 8. And 5-3 is not a horrible result for >the machines. Unfortunately, based on prior experience, the programs tend to >win early games and lose later ones as the GM players spot weaknesses that >can be exploited. Even IM players comment on the current crop of programs not >doing decently with 'king safety' and how they are all very insensitive to >long-term kingside attacks. Playing on ICC will expose such. If I didn't >tweak things regularly, or add code to handle cases I was not handling before, >I would do horribly on ICC. Because after a while, players find the weaknesses >and then they sit on those over and over and over. Unfortunately programs are >complex enough that fixing one often exposes more... So here's the race. Wekanesses are exposed making the programs suddenly dropping in performance, new versions are relesed doing well again, weaknesses are exposed and so on in a never ending story... This will not stop, I suppose, in the next 10 years or so... :-) >Other opinions are welcomed of course. This is only _mine_.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.