Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Crafty Stats

Author: martin fierz

Date: 22:05:38 04/14/04

Go up one level in this thread


On April 14, 2004 at 19:29:44, Christophe Theron wrote:

>On April 14, 2004 at 12:32:56, Robert Hyatt wrote:
>
>>On April 14, 2004 at 10:21:55, martin fierz wrote:
>>
>>>On April 13, 2004 at 17:00:24, Matthew Hull wrote:
>>>
>>>>On April 13, 2004 at 14:21:07, Christophe Theron wrote:
>>>>
>>>>>On April 13, 2004 at 01:29:02, Russell Reagan wrote:
>>>>>
>>>>>>On April 12, 2004 at 23:07:46, Christophe Theron wrote:
>>>>>>
>>>>>>>Further, wouldn't you just *hate* if I took the fun out of chess programming by
>>>>>>>telling you everything? :)
>>>>>>
>>>>>>My gut feeling is that we would probably be disappointed for the most part. I
>>>>>>bet a lot of us think all of you commercial authors are harboring lots of
>>>>>>magical secrets that can turn an average program into a beast. Something similar
>>>>>>to the improvements you get by going from minimax to alphabeta, or by adding
>>>>>>null-move to an average program, and things like that. Those are very
>>>>>>significant improvements.
>>>>>>
>>>>>>I have received the impression from you and other sources like Ed's webpage that
>>>>>>this is not the case. There are some clever things on Ed's webpage, but for the
>>>>>>most part, it is good ideas based on common sense, and then taking the time and
>>>>>>effort to hammer out every last detail to make an idea work, followed by an
>>>>>>efficient implementation.
>>>>>>
>>>>>>To illistrate the difference between what I think a lot of people would expect
>>>>>>to hear from you if you divulged all of your secrets and what I think we would
>>>>>>really get, consider null-move. Null-move is something that you can add to a
>>>>>>program that uses no forward pruning, and once you spend a small amount of time
>>>>>>getting it to work right, the program suddenly plays like it's on steroids
>>>>>>(relatively speaking). However, if we took an average program and added in a few
>>>>>>ideas from Ed's webpage, I wouldn't expect nearly as big of an improvement. I
>>>>>>think you guys just take a lot of ideas and get small improvements here and
>>>>>>there, and at the end of the decade, it amounts to a big improvement. 10%
>>>>>>reduction in tree size here, 20% there, it adds up.
>>>>>>
>>>>>>Am I right? If we are expecting to see magical earth shattering secrets, would
>>>>>>we be disappointed?
>>>>>
>>>>>
>>>>>
>>>>>I don't think you would be disappointed.
>>>>>
>>>>>But you are right in assuming that you would not see a dramatic improvement such
>>>>>as the one you get from alpha-beta vs minimax.
>>>>>
>>>>>You know, one has to wonder where the difference in elo strength between Crafty
>>>>>and the top commercial comes from.
>>>>
>>>>
>>>>Compare this with your mileage at home.  Many of the plus performance scores are
>>>>against accounts running commercial programs.
>>>
>>>
>>>this is irrelevant to the discussion - crafty on ICC is running on 4 processors.
>>>big hardware difference...
>>
>>Nope.  Dual xeon 2.8 with hyperthreading on.  There are faster duals on ICC
>>running the "deep programs."
>>
>>
>>>
>>>it's just what christophe was writing about: crafty is competitive because it
>>>can use multiple processors (x4 = ~100 rating points). on single processor PCs,
>>>it is not competitive with top commercials.
>>>
>>>the reason is that the commercial programmers write a program for the user who
>>>buys it - and this user has 1 processor, with *very* few exceptions. if bob had
>>>spent his time on eval, he would be quite competitive too on single processors
>>>IMO, but that is not what he chose to do.
>>>
>>>i'm speculating here, but probably it's easier for him to focus on
>>>multiprocessing than on knowledge&search for "political" reasons. when you're
>>>paid by the university to do research, they want you to do something that the
>>>people in charge perceive as "useful". tweaking an evaluation function would
>>>probably sound less useful than accomplishing the parallelization of a complex
>>>program. parallelization is also interesting for computer science students,
>>>because the future will bring more multiprocessing systems - also because of
>>>things like  hyperthreading on one processor.
>>>
>>>cheers
>>>  martin
>>
>>Actually UAB doesn't care.  I focus on parallel search because that is actually
>>what interests _me_.  It's an interesting problem.  In 5 years _everybody_ will
>>have a dual as a single chip will have 2-4 processors on it...
>
>
>
>That's what somebody insisted on telling me in Paderborn, 1999.
>
>Maybe it was Martin, actually I don't remember.

do you mean me? i was never in paderborn :-)

cheers
  martin

>These "5 years from now" are like the horizon. Year after year it's always 5
>years from now.
>
>This is also totally ignoring all the new devices like handhelds, smart phones,
>gaming platforms and so on that will not be MP before 20 years from now.
>
>Writing an efficient MP chess program is certainly an interesting problem. So is
>writing an efficient single processor one, and there is no reason this should
>change anytime soon.
>
>
>
>    Christophe
>
>
>
>
>
>>
>>
>>>
>>>>ICC Stats for Crafty since March 21, 2004
>>>>
>>>>Blitz
>>>>   Account      win     loss    draw     pctg
>>>>-- ------------ -----   -----   -----   ------
>>>>br Deveraux     0       1       0         0.00
>>>>br SinbadGonnaD 0       3       0         0.00
>>>>br giant        0       1       1        25.00
>>>>br glories      0       1       1        25.00
>>>>br Bitpusher    1       6       9        34.38
>>>>br BountyHunter 1       3       4        37.50
>>>>br ajop2        1       1       0        50.00
>>>>br allAdreamOfA 1       1       1        50.00
>>>>br bookbuilder  2       2       3        50.00
>>>>br Joecreek2004 0       0       1        50.00
>>>>br Lindisfarne  1       1       2        50.00
>>>>br NubianMagic  0       0       1        50.00
>>>>br Somnus       1       1       0        50.00
>>>>br TheBigChill  1       1       1        50.00
>>>>br Vangard      1       1       0        50.00
>>>>br pathologist  5       4       3        54.17
>>>>br X-Engine     12      5       13       61.67
>>>>br AmazingGrace 19      10      8        62.16
>>>>br Dhaka        2       1       1        62.50
>>>>br PostModernis 8       2       4        71.43
>>>>br stormx       4       0       5        72.22
>>>>br ajop         2       0       2        75.00
>>>>br SearcherX    3       0       3        75.00
>>>>br tlg          4       1       1        75.00
>>>>br muse-comp    7       1       2        80.00
>>>>br Amateur      2       0       0       100.00
>>>>br Clooby       7       0       0       100.00
>>>>br cro-magnon   1       0       0       100.00
>>>>br HangerOn     1       0       0       100.00
>>>>br Nutibara     1       0       0       100.00
>>>>br rigacombinat 2       0       0       100.00
>>>>br TAL9000      2       0       0       100.00
>>>>
>>>>Standard
>>>>
>>>>sr SearcherX    0       1       0         0.00
>>>>sr Vangard      0       1       0         0.00
>>>>sr workuta      0       2       1        16.67
>>>>sr X-Engine     0       1       1        25.00
>>>>sr DIEP         1       1       0        50.00
>>>>sr Good-Boy     1       1       1        50.00
>>>>sr Kronos       0       0       2        50.00
>>>>sr RuffianY     1       1       1        50.00
>>>>sr Sukkubus     2       2       4        50.00
>>>>sr chepla       3       2       4        55.56
>>>>sr HangerOn     1       0       1        75.00
>>>>sr stormx       1       0       1        75.00
>>>>sr thebaron     3       0       1        87.50
>>>>sr BrassCube    1       0       0       100.00
>>>>sr SpiderChessX 1       0       0       100.00
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>On a related note, this brings up a question. If it is true that a lot of things
>>>>>>that give your program improvements at this stage are very minor things, then it
>>>>>>seems logical that those things would not necessarily result in improvements if
>>>>>>they were implemented in other programs, because your ideas probably fit into an
>>>>>>overall system. Do you think it is important to have a good overall system,
>>>>>>where all components compliment one another?
>>>>>>
>>>>>>For instance, a simple example of a system: the job of the full width search is
>>>>>>to hand off nodes to a qsearch, which has the job of handing off quiet positions
>>>>>>to an evaluation function. Under that system, you only want to evaluate quiet
>>>>>>positions, not all positions. If you acheive that, then you make sure your
>>>>>>qsearch is really delivering quiet positions. If it is, you are probably getting
>>>>>>accurate analysis from the engine. If someone took that beefed up qsearch that
>>>>>>was required to make that system work successfully and implemented it in their
>>>>>>program, it may only cause a qsearch explosion and result in weaker play.
>>>>>>
>>>>>>Am I right in believing that it is important to have an overall view of the
>>>>>>system, and that ideas that resulted in improvements in your engine may not help
>>>>>>other engines at all?
>>>>>
>>>>>
>>>>>
>>>>>It is really hard to answer to this question.
>>>>>
>>>>>One thing I am convinced of is that if the top chess programmers started to
>>>>>exchange ideas, like Ed and I did, you would see a significant increase in the
>>>>>strength of these top programs. Clearly some of them would benefit more.
>>>>>
>>>>>
>>>>>
>>>>>    Christophe



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.