Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Best Engine - Real Test :-)

Author: Christophe Theron

Date: 22:07:52 04/16/98

Go up one level in this thread


On April 16, 1998 at 07:09:44, Ricardo Sant Ana wrote:

>Hello All
>
>    Well, yesterday I was talking a friend who really like chess and who
>use computer programs for helping. At some point of our talking, when I
>said I have most of the better programs (well, I would say I have all of
>them), he pointed out that he would be the one who would give most
>attention to chess engines, not to ssdf or search list programs(there
>are lot of conflicts about the trust). So, I remember my friend about
>John Nunn match idea (in chessbase page).But my friend pointed that
>these positions would give fritz better chances, because chessbase set
>the positions for fritz programs and so. I can't say that this is truth
>or not.(and this is not the point) . So,I thought:  why don't we choose
>the positions: there are specialists here, people who works with chess
>programs. We could select some positions: not only one kind of position:
>closed or open, but even strategical lost positions (then we could
>select good defense programs, or even foind out if a program could win a
>strategical winning game).I have other ideas for chess positions too,
>but this is not the case now to talk about all these ideas (for now!).
>Then after choosing this positions (which will take time , because I do
>not believe that democracy came with discution - but of course we will
>get a good solution: most of the programmers of best chess programas are
>here...so they will help choosing positions too) we could choose
>programs and hardwares, and make programs play each other HAND operated
>( I am tired to listen that CASE in ssdf list) . I am developing a new
>homepage, and I will put everything, everynews about this in there since
>I get some time( maybe, just one more weekend).What do you think ? Ins't
>this list about computer chess ? So, let's find out what is best engine
>! (and not better book chess program) All we need now is people who have
>chess programs and 2 computers at least (remember you can't use one
>computer to run two chess programs) and I need tecnichal
>assistance(because I have the programs and some people to help me, but I
>need chess programmers to tell me what is right to do when playing
>computer x computer) :-)
>Well, I really would like people to discuss this idea! and give
>suggestions. I will write more about it.
>I would like people who answer this, asnwer me directly to my e-mail
>too: Because sometimes I just can't access internet explorer but I can
>read e-mails:
>ricardo@aquarius.ime.eb.br
>
>Thanks in advance,
>Ricardo Sant'Ana
>Psy (ICC)

Good idea.

As some talented people pointed out here, the SSDF way of testing is
good because they test a chess program as a whole. The engine, but also
the book, the learning algorithm, the pondering on opponent's time
algorithm, and so on.

But as some other talented people pointed out too, some users are only
interested in knowing which is the best "pure" chess engine, excluding
the bells and whistles around it.

Why? Because some users want their favorite program to ANALYZE
positions, not play games from first to last move. For them, the book is
not that important. The learning feature is unnecessary. Thinking on
opponent's time is useless.

I'm not saying a good program can do without those features. I say that
some users would prefer a low ranked program on the SSDF list if they
know this program has a very good engine, regardless the other features.

So the idea of testing programs without their books, on predefined
positions, sounds interesting to me.

I would suggest that you run the games on only ONE computer (switching
back and forth under W95). The reasons are:
   a) you don't need to test thinking on opponent's time
   b) you will be sure both programs are given the same resources
   c) you will get more volunteers

I know this can be a problem with some Windows programs that "steal" CPU
time even when they are in the background (this is the case of Genius5
for Windows).

In that case, test Windows programs with 2 identical computers. But you
can safely test DOS programs on only ONE computer, which is the best way
in my opinion.

I would also suggest that you don't allow chess programmers to propose
the starting positions. I would even suggest that you decide SECRETLY
(with email correspondance) a set of starting positions, and publish
them only when the first tournament is finished. So you can figure what
happens before and after the programmers begin to tune their programs on
your positions.


    Christophe



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.