Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: How to setup fair and reasonable enginematches?

Author: Will Singleton

Date: 13:29:11 07/25/02

Go up one level in this thread


On July 25, 2002 at 14:40:06, Joachim Rang wrote:

>I'm going to make some automatches between amateur engines. As this is the first
>time I try to make this, I'd be glad to get some advices.
>
>I will use a PIII-Notebook at 1.1 GHz with 256 MB RAM. I plan to make not a
>tournament but one to one matches with much games (a few hundreds) to get
>significant results.
>
>What are fair and reasonable testconditions?
>
>I will run the match on PIII 1.1 GHz and 256 MB RAM under WinXP. I think its
>reasonable to give every engine 80 MB Hashtablesize. I still don't know which
>GUI I shall use. I tested Arena and it worked, but with some problems. With
>Winboard I don't know how to make automatches, maybe someone can help?
>
>First engine will be Yace 0.99.56, which I consider the strongest amateurengine
>today.
>Which engine can compete with Yace?
>
>I think I will run the games with 10 minutes and 10 s per move for each engine.
>Is this to short?
>
>Another question is, which opening books I shall use. Yace has an own book as
>most of the other engines. But I don't know how good they are, SOS for example
>comes only with a very little own book, which will be a handicap. Does someone
>know a "neutral" but large book, which will run with the most engines?
>
>So far today, I'd appreciate any suggestions from you.


For your purpose, Winboard should work just fine.  Just include on the command
line the time control and the number of games (-tc 10 –inc 10 –mg 50).  I don’t
think you’ll get 80mb per engine, maybe 40.  Make sure ponder is off.

I have not been able to get Arena to work well, but it’s certainly a very
promising choice for the future.  For tournaments, you might try WBTM (
http://koti.mbnet.fi/~jorio/tourney/ ), and I believe there’s also something
called Galis WBTM.

I’d use the book that comes with each program.  I can see why you’d want to
normalize the books, but it’s impossible in practice.  Not every program comes
with a bookmaking utility, and even if they did, they all would generate
different lines based on their selection criteria.  It’s possible that one of
the commercial interfaces which support winboard engines would somehow use the
same book for both engines, but someone else will have to comment on that.

Will




This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.