Author: Robert Hyatt
Date: 06:29:53 06/21/99
Go up one level in this thread
On June 21, 1999 at 04:18:57, Jouni Uski wrote: >After this 2 - 2 (Cilkchess removed) against super GMs with average ELO >2621 I think there is no doubt, that PC programs are playing at GM level! >Actually hardly no further games are needed. And remember, that average GM has >only 2500 as rating. That is incorrect. 2500 is the _minimum_ rating to earn the GM title, the average is well above this. > >I was at the playing hall in Paderborn. The conditions were excellent and >sure GMs were playing very serious. I was sitting quite near Fritz - Sokolov >board and after 14. Bh6 Sokolov's look was "interesting". He starts to think >about 20 minutes. In the continuation I didn't heard him to resign, but after >move 24. he simply starts to discuss with Friz operator Feist.... Let me give you my reasons why I _still_ consider computers to be 2400 players: 1. Pick any of the 5 programs that played the GM players. I will find a game where they played so badly that if you look at _that_ game no one would consider that program to be a GM. For example, take the winner and look at the playoff game. Three different GM players commented that they had _never_ seen white screw up the opening so badly... 2. Fritz won in the same way that Deep Blue won the last game vs Kasparov, actually even worse... The GM simply made a blunder that a 2000 rated player would spot in an instant. If you take that win away, the even result goes away. 3. I've been working on chess programming for a long time. And regardless of how they 'seem' to play in many games, I still know just what they can and can't do. And they are nowhere near a GM's level in 'knowledge'. They are still surviving on tactics. And there are plenty of GM players that know how to squelch tactics and make the game hinge on positional play. And there the programs simply don't measure up. 4. I still consider computers to be 2400+. What is the probability that in 5 games, 2400 players could play evenly with a group of GMs rated 2600? some- thing like 25%? 5. GM players exhibit a consistency in quality that computers don't. A computer will play like a GM for 5 games, and like a beginner for 1. What happens when the GM players learn what the computer can't do and then exploit that game after game?. We had at least 4 different GM players watching part of the games on ICC (particularly the GM games the last day) and it was quite a common question "how can the machine play this opening, and then play _that_ move?" Or "why does the machine play so nicely for several games and then play like a beginner here?" So they are very strong, yes. And if you want to lump these games together with what we have so far, I don't have a problem with that. So far, I am counting: Humans 5 (counting Anand and Rohde vs Rebel) progs 3 at 40/2hr. That is within reason for a 2400-2600 rating split, particularly when the just-finished 5 games were played with little preparation by the GM players, since they didn't know who they were playing 24 hours before they played. Yes, I think the machines are doing well. No I don't think they are 2500+ _yet_. And the gaps in the program's knowledge are big enough that I don't think that tactics alone will make up the difference. It goes a long way, but it doesn't close all the holes. One example (I won't name the GM because he asked me not to, and I won't name the program because I am not certain of the hardware nor that the program was that which was claimed since I only watched): The other nite on ICC, a GM friend of mine asked me to watch 2 games a friend of his was playing against a computer. The computer hardware was supposedly a PIII/500 xeon (1 processor) running a program that everyone here thinks is a very 'smart' one based on NPS. In the two games I watched, the GM made it look easy. This 'smart' program seemed to not be real aware of the dangers of a 'distant passed pawn' and allowed the GM to create one, and trade material, until it would see it was lost. I wasn't playing the game, obviously, so I can't say how 'hard' the GM was 'thinking', but my 'friend' (not the one that was playing) commented on this. That is a _fatal_ mistake were this program trying to make its way up the GM ranks, because word spreads, and it simply gets rolled game after game. I am pretty sure that until a GM finds this 'hole', that program will give them fits on decent hardware. And I am also sure that once a GM finds that hole, it is over except for the occasional tactical mistake he will make that the program won't overlook. The thing we are 'missing' in the programs is the 'consistency' that GM players have. IE Shredder played very well throughout the tournament, yet reached a position in the playoff that I doubt even a 2000 player would reach. almost zero mobility. Cramped. Inactive. A position unlike one I have ever seen while watching hundreds of GM games on the chess servers... The problem is _not_ impossible to solve. But it is _not_ solved just yet. This match was 'data'. Let's see how it looks when our data reaches 10 and then 20 games. So far we only have 8. And 5-3 is not a horrible result for the machines. Unfortunately, based on prior experience, the programs tend to win early games and lose later ones as the GM players spot weaknesses that can be exploited. Even IM players comment on the current crop of programs not doing decently with 'king safety' and how they are all very insensitive to long-term kingside attacks. Playing on ICC will expose such. If I didn't tweak things regularly, or add code to handle cases I was not handling before, I would do horribly on ICC. Because after a while, players find the weaknesses and then they sit on those over and over and over. Unfortunately programs are complex enough that fixing one often exposes more... Other opinions are welcomed of course. This is only _mine_.
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.