Author: Don Dailey
Date: 11:28:48 10/06/98
Go up one level in this thread
On October 06, 1998 at 10:33:46, Tim OLena wrote: >I am interested in how programs determine their own ratings, and how accurate >these numbers are in the opinion of those here. For those unfamiliar, Socrates >allows you to tell it the rating you want and presumably it tries to acheive it. >On my old 50 MHz '486, Socrates needed 7 minutes to hit 2400 (the highest number >it will accept). On my new 300 MHz Pentium II, this only takes 30 seconds. Also, > CM6000 reports it's strength as 2651 on my new machine. I wonder if Socrates, >being an older program, is actually playing better than it thinks? And what's >to keep a program such as this from "reading" its environment and "deciding", >"gee, I can play at 2950 on this system and only take two minutes per move! >Maybe I should offer the user this?" So, I'm curious as to why Socrates cut off >at 2400... > >CM6000's rating of the fictional personalities also seems dynamic: the >Chessmaster personality and all fictional personalities come in three points >higher on a system I tried at work. Interestingly, this system is a bit slower >than mine, "only" 266 MHz . . . > >Thanks in advance for your input! > >-TO' Hi Tim. I can tell you how we did the Socrates level but it will be based on my old memories of how I implemented it. We designed it such that it would play the same strength on any computer, it will just take longer on a slow one as you have discovered. If I am remembering it correctly, we calculated the number of nodes required to reach a given rating. We estimated the rating (tried to make it as close to accurate as possible) as some level and extrapolate from there. We know that if you double the speed (or node counts) of a chess program you will get about a 60 rating point improvement so you can calculate any level from this. I'm not sure if we assumed 60 rating points per doubling or something close to this. Why did we stop at 2400? I don't remember. Pehaps at the time we figured it would need too much time to play at this level. We also thought this feature would be good for players trying to improve. You play the 1500 level and when you beat it in a match you can move up to 1550 or 1600. We made an educated guess at the actual rating. The important thing we wanted to do was make the improvements realistic, 2000 will play just about 100 rating points stronger than 1900. If you disagree with the calibration, you can just add or subtract your own adjustment factor to it and it should be reasonable accurate across the whole range. Larry Kaufman did the guesswork and determined what he thought the most realistic formula should be for getting an accurate measurment based on the range of strengths we were shooting for. - Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.