Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Measuring Program Improvement

Author: Robert Hyatt

Date: 06:39:54 08/03/98

Go up one level in this thread


On August 03, 1998 at 03:17:48, Howard Exner wrote:

>What is the typical "growth chart" of a chess program? Does improvement
>taper off after awhile or is it linear? I think programmers have an uncanny feel
>for how their child (chess program) is growing. How do programmers determine
>this growth?

It is rapid at first, but slows once all the normal "bells and whistles" are
in the search.  The "groth" is then followed by a series of ups and downs,
downs as new things are added but with bugs, ups as the bugs are removed to let
the new code perform as expected.

You can measure tactical skill with a quick run through WAC, to see if you solve
more, faster.  Positional changes are harder, because it might hurt tactical
skill but improve play overall.  I measure changes by watching on ICC, looking
at losses, and figuring out the "trend or pattern" in the losses.  I attempt
to plug that evaluation (or search) hole, and then watch for the next pattern,
which should be different.

I also rely on comments of very strong players that play Crafty all the time,
both GM, IM, and non-titled.  That can be more revealing than anything, although
I generally trust my ability to figure out where it is going wrong, just so I
have more than a single position to look at.



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.