Author: Mark Young
Date: 10:54:54 09/01/98
Go up one level in this thread
On September 01, 1998 at 10:19:05, Amir Ban wrote: >On August 31, 1998 at 17:57:02, Mark Young wrote: > >>Is computer Vs computer testing now useless in gauging a chess program’s >>strength playing humans? When Crafty gets killed playing Junior 5 by a wide >>margin. And Fritz 5 draws a match with Rebel 10 even when Rebel 10 has a 2x >>hardware advantage. > >Sounds like you are saying that comp-comp testing show such great rating >differences that they can't possibly be realistic. But the results are skewed. Even in the SSDF testing if one program can get one-sided results against one or two programs it will suggest a bigger advantage against humans, Because it is #1 on the list. And with only a handful of programs being near the top of the list there is not the pool size to correct the over inflated rating. I am not saying that fast searches can not be the best at playing humans. I find fritz 5 to be the best program I have tested so far playing humans. But if computer Vs computer testing squashes other programs that may be better at playing humans or just depresses the rating so much that the general public will pass by the other programs that would also play very well against humans seems very counter productive to the computer chess community as a whole. And I only say this because the reason for computer Vs computer testing was to give the public an idea of which programs are stronger and which ones are weaker in general overall strength. Not which programs are best playing each other but excluding humans. I don’t think this an issue of fast searches Vs slow searches when it comes to playing humans. I think it has been shown that either concept can play very well against strong humans. This is not true, and you >should look less at one or two lopsided results like the above, and more at the >general picture as it is reflected in say the SSDF lists. Remember that Fritz 5 >had some very one-sided results against some opponents, but the bottom line is >that it is 40-50 points ahead of the rest which is not hard to accept as >realistic. The list historically shows a gradual progress and a dependence of >ratings on CPU power which I think is quite easy to believe and to relate to the >actual playing power of programs. > > >>Is it time to abandon Computer Vs Computer testing all >>together? Or are we going to have two standards to judge chess programs? One >>chess program being the best playing other chess programs and one chess program >>being the best playing humans. > >Playing against computers and playing against humans are different experiences. >No argument there. But to say that strength vs. computers and strength vs. >humans is unrelated, or even to say that there are some "weak" programs who are >actually stronger when it comes to humans is not supported in fact. Actually the >facts indicate the opposite. > >It has always been the strongest programs who have also scored best against >humans. And vice versa: there is no example of a program that is not considered >to be at the top but that is scoring better against humans (than the top >programs). Someone mentioned CSTal on this thread as "proving" this point, which >strikes me really odd, since I've never heard of any achievement of this program >against a strong human. Yes very odd. I have seen no achievement for this program playing human or computer. > >Amir
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.