Computer Chess Club Archives


Search

Terms

Messages

Subject: Crafty Stats

Author: Matthew Hull

Date: 14:00:24 04/13/04

Go up one level in this thread


On April 13, 2004 at 14:21:07, Christophe Theron wrote:

>On April 13, 2004 at 01:29:02, Russell Reagan wrote:
>
>>On April 12, 2004 at 23:07:46, Christophe Theron wrote:
>>
>>>Further, wouldn't you just *hate* if I took the fun out of chess programming by
>>>telling you everything? :)
>>
>>My gut feeling is that we would probably be disappointed for the most part. I
>>bet a lot of us think all of you commercial authors are harboring lots of
>>magical secrets that can turn an average program into a beast. Something similar
>>to the improvements you get by going from minimax to alphabeta, or by adding
>>null-move to an average program, and things like that. Those are very
>>significant improvements.
>>
>>I have received the impression from you and other sources like Ed's webpage that
>>this is not the case. There are some clever things on Ed's webpage, but for the
>>most part, it is good ideas based on common sense, and then taking the time and
>>effort to hammer out every last detail to make an idea work, followed by an
>>efficient implementation.
>>
>>To illistrate the difference between what I think a lot of people would expect
>>to hear from you if you divulged all of your secrets and what I think we would
>>really get, consider null-move. Null-move is something that you can add to a
>>program that uses no forward pruning, and once you spend a small amount of time
>>getting it to work right, the program suddenly plays like it's on steroids
>>(relatively speaking). However, if we took an average program and added in a few
>>ideas from Ed's webpage, I wouldn't expect nearly as big of an improvement. I
>>think you guys just take a lot of ideas and get small improvements here and
>>there, and at the end of the decade, it amounts to a big improvement. 10%
>>reduction in tree size here, 20% there, it adds up.
>>
>>Am I right? If we are expecting to see magical earth shattering secrets, would
>>we be disappointed?
>
>
>
>I don't think you would be disappointed.
>
>But you are right in assuming that you would not see a dramatic improvement such
>as the one you get from alpha-beta vs minimax.
>
>You know, one has to wonder where the difference in elo strength between Crafty
>and the top commercial comes from.


Compare this with your mileage at home.  Many of the plus performance scores are
against accounts running commercial programs.

ICC Stats for Crafty since March 21, 2004

Blitz
   Account      win     loss    draw     pctg
-- ------------ -----   -----   -----   ------
br Deveraux     0       1       0         0.00
br SinbadGonnaD 0       3       0         0.00
br giant        0       1       1        25.00
br glories      0       1       1        25.00
br Bitpusher    1       6       9        34.38
br BountyHunter 1       3       4        37.50
br ajop2        1       1       0        50.00
br allAdreamOfA 1       1       1        50.00
br bookbuilder  2       2       3        50.00
br Joecreek2004 0       0       1        50.00
br Lindisfarne  1       1       2        50.00
br NubianMagic  0       0       1        50.00
br Somnus       1       1       0        50.00
br TheBigChill  1       1       1        50.00
br Vangard      1       1       0        50.00
br pathologist  5       4       3        54.17
br X-Engine     12      5       13       61.67
br AmazingGrace 19      10      8        62.16
br Dhaka        2       1       1        62.50
br PostModernis 8       2       4        71.43
br stormx       4       0       5        72.22
br ajop         2       0       2        75.00
br SearcherX    3       0       3        75.00
br tlg          4       1       1        75.00
br muse-comp    7       1       2        80.00
br Amateur      2       0       0       100.00
br Clooby       7       0       0       100.00
br cro-magnon   1       0       0       100.00
br HangerOn     1       0       0       100.00
br Nutibara     1       0       0       100.00
br rigacombinat 2       0       0       100.00
br TAL9000      2       0       0       100.00

Standard

sr SearcherX    0       1       0         0.00
sr Vangard      0       1       0         0.00
sr workuta      0       2       1        16.67
sr X-Engine     0       1       1        25.00
sr DIEP         1       1       0        50.00
sr Good-Boy     1       1       1        50.00
sr Kronos       0       0       2        50.00
sr RuffianY     1       1       1        50.00
sr Sukkubus     2       2       4        50.00
sr chepla       3       2       4        55.56
sr HangerOn     1       0       1        75.00
sr stormx       1       0       1        75.00
sr thebaron     3       0       1        87.50
sr BrassCube    1       0       0       100.00
sr SpiderChessX 1       0       0       100.00


>
>
>
>
>>On a related note, this brings up a question. If it is true that a lot of things
>>that give your program improvements at this stage are very minor things, then it
>>seems logical that those things would not necessarily result in improvements if
>>they were implemented in other programs, because your ideas probably fit into an
>>overall system. Do you think it is important to have a good overall system,
>>where all components compliment one another?
>>
>>For instance, a simple example of a system: the job of the full width search is
>>to hand off nodes to a qsearch, which has the job of handing off quiet positions
>>to an evaluation function. Under that system, you only want to evaluate quiet
>>positions, not all positions. If you acheive that, then you make sure your
>>qsearch is really delivering quiet positions. If it is, you are probably getting
>>accurate analysis from the engine. If someone took that beefed up qsearch that
>>was required to make that system work successfully and implemented it in their
>>program, it may only cause a qsearch explosion and result in weaker play.
>>
>>Am I right in believing that it is important to have an overall view of the
>>system, and that ideas that resulted in improvements in your engine may not help
>>other engines at all?
>
>
>
>It is really hard to answer to this question.
>
>One thing I am convinced of is that if the top chess programmers started to
>exchange ideas, like Ed and I did, you would see a significant increase in the
>strength of these top programs. Clearly some of them would benefit more.
>
>
>
>    Christophe



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.