Author: Mark Young
Date: 11:35:24 09/01/98
Go up one level in this thread
On September 01, 1998 at 13:36:46, Robert Hyatt wrote: >On September 01, 1998 at 12:55:33, Tim Mirabile wrote: > >>On September 01, 1998 at 06:31:39, Jouni Uski wrote: >> >>>So far I haven't seen any statistical proof, that one program is better against >>>human than against comp (or reverse). >> >>It all comes back to the same thing. We don't have enough comp-human games at >>slow time controls, compared to 60000+ comp-comp games played by SSDF, to make >>any kind of comparison. So just because you haven't seen the proof, doesn't >>mean it's not true, and of course this also does not mean that it is true. > > >Here's one interesting bit of data.. during the month of July, and early >August, I had a pretty serious bug in crafty's eval function. During this >time, crafty was not able to win even 1 of every 20 games against computers, >yet it was having the same 80% win ratio against the usual group of GM >challengers that it always has... And it plays so many more games against >GM players than against computers that this went undetected for quite a >while... > >The question is, why did this happen? Obviously the eval bug was seriously >affecting its play, because I saw game after game where it would make a >move and see things fall apart, when playing a computer, but against >humans it didn't happen. So clearly there is some significant difference >between playing a computer and a human. My speculation is that the tactical >skill of a computer is so good that any little bit of bogus analysis will >eventually lead to problems, while against a human, mistakes deep in the >tree are not nearly as important as how the position is "preserved" through >the game... > >The bug has been fixed, scores against computers are now much better, yet >against GM's it has not changed appreciably at all... I agree we need more data, but because of this crafty bug and how much it did differ in performance against humans and computers has to raise a few alarm bells in our faith of computer Vs computer testing. Who is to say that there could not be a program that could perform wonders playing humans, but could not win a game against computers, without having a bug. The Crafty results makes computer Vs computer testing very suspect in my eyes. If the goal of chess programming is still to beat top human players.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.