Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Engine Testing

Author: Dann Corbit

Date: 16:59:56 01/03/06

Go up one level in this thread


On January 02, 2006 at 07:50:02, Ted Summers wrote:

>I have been viewing this forum for some time now. Even though I am no longer
>into writing code, I am interested in beta testing and creating opening books. I
>have two major hobbies in life and they are chess and computers. However
>computers and networking pay my bills so it is easy to guess which one gets most
>of my time. Having said that I no longer play any real active chess, but I hover
>somewhere between class A and a low Expert player (USCF) rating wise. So what do
>the experts here think it takes to do any real beta testing? These are the
>points I would like you to tell me about:
>
>* Opening suites
>* Endgame suites
>* Middle suites (?)

EPD test suites follow the EPD subset of the PGN standard.  See (for instance):
http://www.very-best.de/pgn-spec.htm

Test suites show an engine's tactical ability, but tell little about strength of
play.

>* Opening Book Creation

There are two sorts of opening books.  There are professional books written by
chess experts and (generally automated) amateur books composed by hobbyists.

The professional books are generally much stronger and have fewer defects.

>* Engine Testing

Engine testing is performed by a number of individuals and organizations.
Among the best testing efforts currently active are the SSDF, CEGT and WBEC.  A
web search will turn up the relevant pages.

>* Operating Systems

The vast majority of PC chess programs are run on Windows systems.  After that,
Linux and Mac platforms.

>* PC Hardware

AMD 64 bit systems are very good for most chess programs.

>* Testing Time

People test at all different times.
The slowest time control is 40 moves in 2 hours (used by SSDF).
A more common time control is 40 moves in 40 minutes (for example, used by
CEGT).

>* Autoplayer

Autoplayer is an add-on that allows you to play two different machines against
each other over a serial port.  It seems that just playing over the net would be
much more natural in this day and age, but RS232 support is very popular for
some reason.

>* Ponder Issues

If you only have one machine with one CPU, then ponder testing is right out.
You need at least 2 CPUs if you are going to let the programs think all the
time.

>* Online Testing

Basically, only results are found online.  You can watch some test sessions live
once in a while.

>* Offline Testing

This is how it is done, basically.

>* Engine GUI

http://www.aarontay.per.sg/Winboard/interface.html

>* Anything Related but not listed

General chess engine information:
http://www.aarontay.per.sg/Winboard/
http://www.uciengines.de/

Testing stuff:
http://web.telia.com/~u85924109/ssdf/list.htm
http://www.husvankempen.de/nunn/
http://wbec-ridderkerk.nl/
http://www.geocities.com/lyapko/winboard.htm

Read about professional products:
http://www.chessreviews.com/

I buy stuff here:
http://www.chessusa.com/Merchant2/merchant.mvc?Screen=CTGY&Store_Code=YMCG&Category_Code=CHSOFT
http://www.lokasoft.nl/
http://www.convekta.com/

Nifty free stuff:
http://scid.sourceforge.net/
http://jose-chess.sourceforge.net/
The engines links in the WBEC page and the UCI pages up above will point you to
free chess engines.

>Thanks to everyone in advance!

More specific questions will garner more specific answers.



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.