Author: Dann Corbit
Date: 15:24:31 01/06/03
Go up one level in this thread
On January 06, 2003 at 17:40:53, Lieven Clarisse wrote: >On January 06, 2003 at 16:56:35, Tom King wrote: > >>Hi all, >> >>What do people think about playing different versions of your program against >>each other as a way of testing? >> >>I'm playing around with it right now, between v0.07 and a newer version of my >>program. The newer version is winning handsomely: +24,=18,-10. >> >>This implies a reasonably impressive increase in strength, almost 100 ELO. Ok, >>ok, it's a small sample, so the margin of error could be big. >> >>However, my gut feel is that playing different versions of your programs tends >>to overstate the strength differences. What do people think? >> > >The best way IMHO is to test it against engines with more or less equal >strenght. You can use WBEC ratinglist to get an idea of the strenghts of the >different engines. Try to find a range were your program gets %50 score (for >instance range 40-50 from the ratinglist, when changing your program and see you >get >60% (for a sufficient large numbere of games) it is time to play against >the range 35-45, etc.. Strength is best measured when playing equal opps. I disagree. I think you get better results from two sets: 1. Programs that are about 100 ELO weaker. 2. Programs that are about 100 ELO stronger. When the programs are about the same strength, you get too much coin toss effect.
This page took 0.22 seconds to execute
Last modified: Thu, 07 Jul 11 08:48:38 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.