Author: James Swafford
Date: 16:52:59 11/16/01
Go up one level in this thread
On November 16, 2001 at 18:46:35, Brian Richardson wrote: >I am somewhat frustrated. I made some changes to Tinker adding lazy eval and >some other terms and then tested several things. > >It improved nps speed by about 30%. > >It improved WAC results. > >It won or drew every self play game (blitz and standard). > >Then I put the new version up to play at ICC. >It promptly lost 100 points in blitz and standard !? > >This is not the first time that my "enhancements" seemed to improve things but >turned out to play worse in actual games. > >Does this sort of thing happen to others too? > >Thanks >Brian I think ICC ratings are very useful, but consider the noise in them. How likely is it that the drop in ratings doesn't reflect the strength added by your enhancements? Pretty likely actually. I'm developing a program that logs the results of every 5 3 r game. Every new version starts a new game log. Additionally, I have a perl script on the same machine that parses the game log and plots the rating as a function of time (or games played). After a sufficiently large number of games, the mean rating will tell the real story. Other things are useful, too (median, min, max, standard deviation), but I think the mean rating over a couple hundred games is pretty indicative. Eyeballing things can be pretty dangerous (and frustrating). -- James
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.