Author: Vincent Diepeveen
Date: 03:05:08 06/28/98
Go up one level in this thread
On June 27, 1998 at 22:52:13, Robert Hyatt wrote: >On June 27, 1998 at 19:28:01, Mark Young wrote: > >> >>Are their any test suites that can give you an idea of how strong a program >>really is? The more I run test suites today, the more I am convinced that they >>are meaningless in gauging a program s strength > >no. several have tried, but running a suite, then fitting a bizarre function >to known ratings of programs that can be used to produce results for that >suite. But this doesn't mean a thing to *other* programs that are different >from the ones used to create the formula. > >Ask Don about Larry's formula from an old suite of his that he thought was >pretty good, *until* we ran the suite on Cray Blitz at Indianapolis ACM (I >think). It blew out the formula badly... and gave results that were just >about meaningless. because it solved almost everything in 0 seconds per >position... >much better to do as you (and I and Bruce and others) do and play them on a >server against known humans and programs. That gives a much better feel than >a series of chess positions. I completely agree with Bob, test suites are nice to test some tactics, but not the strength of a program. Agressive programs also score way better on test suites than others, so after a suite you do not even know how strong they tactical are relatively to each other. Way better is to play games. Note that the limited depth you get in blitz on the internet at todays payable hardware is quite limited. So making it play tactically well might pay for blitz.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.