Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Testing Chess Programs

Author: Christophe Theron

Date: 11:21:07 04/13/04

Go up one level in this thread


On April 13, 2004 at 01:29:02, Russell Reagan wrote:

>On April 12, 2004 at 23:07:46, Christophe Theron wrote:
>
>>Further, wouldn't you just *hate* if I took the fun out of chess programming by
>>telling you everything? :)
>
>My gut feeling is that we would probably be disappointed for the most part. I
>bet a lot of us think all of you commercial authors are harboring lots of
>magical secrets that can turn an average program into a beast. Something similar
>to the improvements you get by going from minimax to alphabeta, or by adding
>null-move to an average program, and things like that. Those are very
>significant improvements.
>
>I have received the impression from you and other sources like Ed's webpage that
>this is not the case. There are some clever things on Ed's webpage, but for the
>most part, it is good ideas based on common sense, and then taking the time and
>effort to hammer out every last detail to make an idea work, followed by an
>efficient implementation.
>
>To illistrate the difference between what I think a lot of people would expect
>to hear from you if you divulged all of your secrets and what I think we would
>really get, consider null-move. Null-move is something that you can add to a
>program that uses no forward pruning, and once you spend a small amount of time
>getting it to work right, the program suddenly plays like it's on steroids
>(relatively speaking). However, if we took an average program and added in a few
>ideas from Ed's webpage, I wouldn't expect nearly as big of an improvement. I
>think you guys just take a lot of ideas and get small improvements here and
>there, and at the end of the decade, it amounts to a big improvement. 10%
>reduction in tree size here, 20% there, it adds up.
>
>Am I right? If we are expecting to see magical earth shattering secrets, would
>we be disappointed?



I don't think you would be disappointed.

But you are right in assuming that you would not see a dramatic improvement such
as the one you get from alpha-beta vs minimax.

You know, one has to wonder where the difference in elo strength between Crafty
and the top commercial comes from.




>On a related note, this brings up a question. If it is true that a lot of things
>that give your program improvements at this stage are very minor things, then it
>seems logical that those things would not necessarily result in improvements if
>they were implemented in other programs, because your ideas probably fit into an
>overall system. Do you think it is important to have a good overall system,
>where all components compliment one another?
>
>For instance, a simple example of a system: the job of the full width search is
>to hand off nodes to a qsearch, which has the job of handing off quiet positions
>to an evaluation function. Under that system, you only want to evaluate quiet
>positions, not all positions. If you acheive that, then you make sure your
>qsearch is really delivering quiet positions. If it is, you are probably getting
>accurate analysis from the engine. If someone took that beefed up qsearch that
>was required to make that system work successfully and implemented it in their
>program, it may only cause a qsearch explosion and result in weaker play.
>
>Am I right in believing that it is important to have an overall view of the
>system, and that ideas that resulted in improvements in your engine may not help
>other engines at all?



It is really hard to answer to this question.

One thing I am convinced of is that if the top chess programmers started to
exchange ideas, like Ed and I did, you would see a significant increase in the
strength of these top programs. Clearly some of them would benefit more.



    Christophe



This page took 0.02 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.