Author: Michel Langeveld
Date: 11:54:15 12/21/01
Go up one level in this thread
On December 21, 2001 at 12:55:50, K. Burcham wrote: > > >what do you mean? >are you saying you would like to see all programs before they >start the tournament, perform this benchmark test? >and for what purpose? to reveal a kns for a particular program >on the mhz and hardware that is used for that tournament that day? > >i think that would be interesting. but most here know pretty close >to the actual kns each program will run on most hardware. > >kburcham Official Benchmark?? Well here are just my 2 cents. Benchmark #1 (Testsuite) ============ Well connect 10 testsuites together (3000 positions or so) and call that benchmark. Make a mixture of: -tactics -strategic -opening -middlegame -endgame -mates -puzzles -etc Calculate all positions on 3 min/move. This means we are in 6 weeks ready. Make an conversion scheme of the hardware to use. Make universal benchmark program which spits out how much thinking time an engine may use to get "correct" benchmark results. % number of correct can be a benchmark then. Benchmark #1B ============ Same as #1 but then knps, avarage depth, avarage time to find solution, evaluations, number of nodes in qsearch, number of fail highes, etc Benchmark #2 (Games) ============ Looks a bit like ssdf: 100 games of the types: -01 minute all moves -05 minute all moves -15 minute all moves -60 minute all moves against 4 well defined and well tested engines : -1000 elo -1500 elo -2000 elo -2500 elo I used 500 elo delta's because 750 is needed to win 100% So an 1900 elo program still doesn't loose all from 2500. This means 4x4x100 = 1600 games as testinput to create a benchmark score. Benchmark #2B ============= Same as #2 but then with no opening books, or with no pondering, or with no hash, or whatever Same as #2 but then at certain middlegame position like the middlegame testsuite. regards, Michel
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.