Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Test Suites

Author: Joshua Lee

Date: 07:32:07 05/22/01

Go up one level in this thread


It is hard to tell if they show actual strength , Fritz 6 on my 800Mhz Athlon
(which now is becomming a dinosaur) should hold it's own if not win a match
against any other program out there however i have put all the test sets
together that i could find except for CAP positions way way too many, removed
doubles and went from 17000 down to around 10,000   . I had fritz6 solve what it
could with 3 minutes to move and solved 8,261. Now i still have 3470 and the
ones fritz wasn't  able to solve with 3 minutes, obviously the file was larger
and i have gone through some positions at 30minutes but fritz still can't solve
those positions while other programs probably can. I cant say for 100% but if
you think about this you have millions of cap positions not included and you go
by the major test sets in which fritz will solve around 75% of these in under
3minutes and countless others not in 30 or more minutes.

The amount of positions is such a small percent of the entire amount of cap
positions and important to know /need to know positions. for 1 to play the
opening you need anywhere from 10 moves to 24 by each side , would the computer
find them maybe if you gave it months or years on a cray. There are 5 eco books
with 100 codes each and within any one you can have 15 or more lines. Maybe
someone knows the exact number of opening positions in eco but then you have
endgame positions but you pretty much have those covered by tablebases and what
isn't in those which are important are only some 6-12man positions i can't say
how much those matter so that leaves middlegame positions. Of test sets out
there do any have just a bunch of critical positions or do any test how the
program treats certain pawn structures etc.

Somebody needs to compile a big set which would give you a good idea based on
performance which would tell you what the program does well and what it doesn't
and what level it performs on.



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.