Author: Dann Corbit
Date: 22:13:25 02/15/01
Go up one level in this thread
On February 16, 2001 at 00:34:16, Aaron Tay wrote: >1) Can someone summarise for me what the main well known test suites are? Here is a part of the list: AEMIS AEMIS2 arasan2 BLOSS BS2830 BT2450 BT2630 BWTC COVAX COVAX1 COVAX2 COVAX3 CRAFTY ECE3 ECM ECM98 ECM98H ECMF5 FINE GMG1 GMG2 GMG3 GS2930 KAUFMAN LCT2 LK Mats MES NOLOT NUNNTEST POSSAC1 POSSAC2 S80 SHEP TWGCG TYPP USKIEG VA WAC WCSAC YAZGAC There are lots and lots more besides these. For info on EPD test suites, try Maro's chess page, Shep's chess page, V. Abillo's chess page, and my ftp site's EPD directory. I recall some page with a title something like "Yellow Chess Club" or the like that had some nice EPD test set stuff. EPD test suites do not show how strong a program is at playing chess. They show how strong it is at solving chess problems. Though there is surely some sort of correlation between the two activities, it is not known how to make the relationship accurately described. An interesting experiment was held by the Rebel folks. A number of different parameters were tried to find the best chess playing settings, the best problem solving settings etc. The best chess playing settings were not the same as the best problem solving settings. >Besides WAC that is..I used to recall a webpage that compared this and someone >help? > >2) Given all the complains about the uneven quality of Rating lists, what >factors would you look for in deciding if a rating list is a quliaty one? > >Eg > >* Number of games >* testing procedures [handling of bugs in book learning etc] >* Quality of testers [Would fewer quality testers be better than lots?] >* Transparancy [availability of games?, policy statements,audits? ] >* Perceived indepedence >* Hardware used > > >any more? And how do the various rating lists either by organisations [SSDF], >e-zines [e-bit, selective search] or single persons [eg Frank Quisinsky's list] >compare acording to the citerias? > >It seems to me that SSDF seems to be the mostly superior in most areas, altough, >they need to work on the perceived indepedence part.. I don't think they can change people's opinion. I think their list is the best one, but you must realize that for this list, it is valid with the exact conditions of the test: 1. Using Autoplayer 2. Using the exact version of the program stated with the book that comes with the program 3. Using the exact version of hardware stated. Under any other conditions, you must assume that things can change.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.