Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Measuring program improvement

Author: Peter McKenzie

Date: 18:19:10 11/16/01

Go up one level in this thread


On November 16, 2001 at 18:46:35, Brian Richardson wrote:

>I am somewhat frustrated.  I made some changes to Tinker adding lazy eval and
>some other terms and then tested several things.
>
>It improved nps speed by about 30%.
>
>It improved WAC results.
>
>It won or drew every self play game (blitz and standard).

This sounds weird, how many did you play?
A meaningful match is typically going to be well over 50 games, are you saying
that the new version won or drew every game in such a match?????

Or did you just play 2 or 3 games, in which case I'd say you were basically
wasting your time.

>
>Then I put the new version up to play at ICC.
>It promptly lost 100 points in blitz and standard !?

ICC ratings really move around alot as you know.  I personally don't rely on
them for much, but if I did I would be doing something such as:  take the
average rating over a series of 200 hundred games.  I would NOT just rely on the
rating AFTER 200 games.

>
>This is not the first time that my "enhancements" seemed to improve things but
>turned out to play worse in actual games.
>
>Does this sort of thing happen to others too?

in general, yes!

>
>Thanks
>Brian



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.