Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Crafty Stats

Author: Robert Hyatt

Date: 09:32:56 04/14/04

Go up one level in this thread


On April 14, 2004 at 10:21:55, martin fierz wrote:

>On April 13, 2004 at 17:00:24, Matthew Hull wrote:
>
>>On April 13, 2004 at 14:21:07, Christophe Theron wrote:
>>
>>>On April 13, 2004 at 01:29:02, Russell Reagan wrote:
>>>
>>>>On April 12, 2004 at 23:07:46, Christophe Theron wrote:
>>>>
>>>>>Further, wouldn't you just *hate* if I took the fun out of chess programming by
>>>>>telling you everything? :)
>>>>
>>>>My gut feeling is that we would probably be disappointed for the most part. I
>>>>bet a lot of us think all of you commercial authors are harboring lots of
>>>>magical secrets that can turn an average program into a beast. Something similar
>>>>to the improvements you get by going from minimax to alphabeta, or by adding
>>>>null-move to an average program, and things like that. Those are very
>>>>significant improvements.
>>>>
>>>>I have received the impression from you and other sources like Ed's webpage that
>>>>this is not the case. There are some clever things on Ed's webpage, but for the
>>>>most part, it is good ideas based on common sense, and then taking the time and
>>>>effort to hammer out every last detail to make an idea work, followed by an
>>>>efficient implementation.
>>>>
>>>>To illistrate the difference between what I think a lot of people would expect
>>>>to hear from you if you divulged all of your secrets and what I think we would
>>>>really get, consider null-move. Null-move is something that you can add to a
>>>>program that uses no forward pruning, and once you spend a small amount of time
>>>>getting it to work right, the program suddenly plays like it's on steroids
>>>>(relatively speaking). However, if we took an average program and added in a few
>>>>ideas from Ed's webpage, I wouldn't expect nearly as big of an improvement. I
>>>>think you guys just take a lot of ideas and get small improvements here and
>>>>there, and at the end of the decade, it amounts to a big improvement. 10%
>>>>reduction in tree size here, 20% there, it adds up.
>>>>
>>>>Am I right? If we are expecting to see magical earth shattering secrets, would
>>>>we be disappointed?
>>>
>>>
>>>
>>>I don't think you would be disappointed.
>>>
>>>But you are right in assuming that you would not see a dramatic improvement such
>>>as the one you get from alpha-beta vs minimax.
>>>
>>>You know, one has to wonder where the difference in elo strength between Crafty
>>>and the top commercial comes from.
>>
>>
>>Compare this with your mileage at home.  Many of the plus performance scores are
>>against accounts running commercial programs.
>
>
>this is irrelevant to the discussion - crafty on ICC is running on 4 processors.
>big hardware difference...

Nope.  Dual xeon 2.8 with hyperthreading on.  There are faster duals on ICC
running the "deep programs."


>
>it's just what christophe was writing about: crafty is competitive because it
>can use multiple processors (x4 = ~100 rating points). on single processor PCs,
>it is not competitive with top commercials.
>
>the reason is that the commercial programmers write a program for the user who
>buys it - and this user has 1 processor, with *very* few exceptions. if bob had
>spent his time on eval, he would be quite competitive too on single processors
>IMO, but that is not what he chose to do.
>
>i'm speculating here, but probably it's easier for him to focus on
>multiprocessing than on knowledge&search for "political" reasons. when you're
>paid by the university to do research, they want you to do something that the
>people in charge perceive as "useful". tweaking an evaluation function would
>probably sound less useful than accomplishing the parallelization of a complex
>program. parallelization is also interesting for computer science students,
>because the future will bring more multiprocessing systems - also because of
>things like  hyperthreading on one processor.
>
>cheers
>  martin

Actually UAB doesn't care.  I focus on parallel search because that is actually
what interests _me_.  It's an interesting problem.  In 5 years _everybody_ will
have a dual as a single chip will have 2-4 processors on it...


>
>>ICC Stats for Crafty since March 21, 2004
>>
>>Blitz
>>   Account      win     loss    draw     pctg
>>-- ------------ -----   -----   -----   ------
>>br Deveraux     0       1       0         0.00
>>br SinbadGonnaD 0       3       0         0.00
>>br giant        0       1       1        25.00
>>br glories      0       1       1        25.00
>>br Bitpusher    1       6       9        34.38
>>br BountyHunter 1       3       4        37.50
>>br ajop2        1       1       0        50.00
>>br allAdreamOfA 1       1       1        50.00
>>br bookbuilder  2       2       3        50.00
>>br Joecreek2004 0       0       1        50.00
>>br Lindisfarne  1       1       2        50.00
>>br NubianMagic  0       0       1        50.00
>>br Somnus       1       1       0        50.00
>>br TheBigChill  1       1       1        50.00
>>br Vangard      1       1       0        50.00
>>br pathologist  5       4       3        54.17
>>br X-Engine     12      5       13       61.67
>>br AmazingGrace 19      10      8        62.16
>>br Dhaka        2       1       1        62.50
>>br PostModernis 8       2       4        71.43
>>br stormx       4       0       5        72.22
>>br ajop         2       0       2        75.00
>>br SearcherX    3       0       3        75.00
>>br tlg          4       1       1        75.00
>>br muse-comp    7       1       2        80.00
>>br Amateur      2       0       0       100.00
>>br Clooby       7       0       0       100.00
>>br cro-magnon   1       0       0       100.00
>>br HangerOn     1       0       0       100.00
>>br Nutibara     1       0       0       100.00
>>br rigacombinat 2       0       0       100.00
>>br TAL9000      2       0       0       100.00
>>
>>Standard
>>
>>sr SearcherX    0       1       0         0.00
>>sr Vangard      0       1       0         0.00
>>sr workuta      0       2       1        16.67
>>sr X-Engine     0       1       1        25.00
>>sr DIEP         1       1       0        50.00
>>sr Good-Boy     1       1       1        50.00
>>sr Kronos       0       0       2        50.00
>>sr RuffianY     1       1       1        50.00
>>sr Sukkubus     2       2       4        50.00
>>sr chepla       3       2       4        55.56
>>sr HangerOn     1       0       1        75.00
>>sr stormx       1       0       1        75.00
>>sr thebaron     3       0       1        87.50
>>sr BrassCube    1       0       0       100.00
>>sr SpiderChessX 1       0       0       100.00
>>
>>
>>>
>>>
>>>
>>>
>>>>On a related note, this brings up a question. If it is true that a lot of things
>>>>that give your program improvements at this stage are very minor things, then it
>>>>seems logical that those things would not necessarily result in improvements if
>>>>they were implemented in other programs, because your ideas probably fit into an
>>>>overall system. Do you think it is important to have a good overall system,
>>>>where all components compliment one another?
>>>>
>>>>For instance, a simple example of a system: the job of the full width search is
>>>>to hand off nodes to a qsearch, which has the job of handing off quiet positions
>>>>to an evaluation function. Under that system, you only want to evaluate quiet
>>>>positions, not all positions. If you acheive that, then you make sure your
>>>>qsearch is really delivering quiet positions. If it is, you are probably getting
>>>>accurate analysis from the engine. If someone took that beefed up qsearch that
>>>>was required to make that system work successfully and implemented it in their
>>>>program, it may only cause a qsearch explosion and result in weaker play.
>>>>
>>>>Am I right in believing that it is important to have an overall view of the
>>>>system, and that ideas that resulted in improvements in your engine may not help
>>>>other engines at all?
>>>
>>>
>>>
>>>It is really hard to answer to this question.
>>>
>>>One thing I am convinced of is that if the top chess programmers started to
>>>exchange ideas, like Ed and I did, you would see a significant increase in the
>>>strength of these top programs. Clearly some of them would benefit more.
>>>
>>>
>>>
>>>    Christophe



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.