Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Crafty Stats

Author: martin fierz

Date: 16:05:25 04/14/04

Go up one level in this thread


On April 14, 2004 at 13:44:54, Robert Hyatt wrote:

>On April 14, 2004 at 10:38:52, martin fierz wrote:
>
>>On April 14, 2004 at 10:33:42, Matthew Hull wrote:
>>
>>>On April 14, 2004 at 10:21:55, martin fierz wrote:
>>>
>>>>On April 13, 2004 at 17:00:24, Matthew Hull wrote:
>>>>
>>>>>On April 13, 2004 at 14:21:07, Christophe Theron wrote:
>>>>>
>>>>>>On April 13, 2004 at 01:29:02, Russell Reagan wrote:
>>>>>>
>>>>>>>On April 12, 2004 at 23:07:46, Christophe Theron wrote:
>>>>>>>
>>>>>>>>Further, wouldn't you just *hate* if I took the fun out of chess programming by
>>>>>>>>telling you everything? :)
>>>>>>>
>>>>>>>My gut feeling is that we would probably be disappointed for the most part. I
>>>>>>>bet a lot of us think all of you commercial authors are harboring lots of
>>>>>>>magical secrets that can turn an average program into a beast. Something similar
>>>>>>>to the improvements you get by going from minimax to alphabeta, or by adding
>>>>>>>null-move to an average program, and things like that. Those are very
>>>>>>>significant improvements.
>>>>>>>
>>>>>>>I have received the impression from you and other sources like Ed's webpage that
>>>>>>>this is not the case. There are some clever things on Ed's webpage, but for the
>>>>>>>most part, it is good ideas based on common sense, and then taking the time and
>>>>>>>effort to hammer out every last detail to make an idea work, followed by an
>>>>>>>efficient implementation.
>>>>>>>
>>>>>>>To illistrate the difference between what I think a lot of people would expect
>>>>>>>to hear from you if you divulged all of your secrets and what I think we would
>>>>>>>really get, consider null-move. Null-move is something that you can add to a
>>>>>>>program that uses no forward pruning, and once you spend a small amount of time
>>>>>>>getting it to work right, the program suddenly plays like it's on steroids
>>>>>>>(relatively speaking). However, if we took an average program and added in a few
>>>>>>>ideas from Ed's webpage, I wouldn't expect nearly as big of an improvement. I
>>>>>>>think you guys just take a lot of ideas and get small improvements here and
>>>>>>>there, and at the end of the decade, it amounts to a big improvement. 10%
>>>>>>>reduction in tree size here, 20% there, it adds up.
>>>>>>>
>>>>>>>Am I right? If we are expecting to see magical earth shattering secrets, would
>>>>>>>we be disappointed?
>>>>>>
>>>>>>
>>>>>>
>>>>>>I don't think you would be disappointed.
>>>>>>
>>>>>>But you are right in assuming that you would not see a dramatic improvement such
>>>>>>as the one you get from alpha-beta vs minimax.
>>>>>>
>>>>>>You know, one has to wonder where the difference in elo strength between Crafty
>>>>>>and the top commercial comes from.
>>>>>
>>>>>
>>>>>Compare this with your mileage at home.  Many of the plus performance scores are
>>>>>against accounts running commercial programs.
>>>>
>>>>
>>>>this is irrelevant to the discussion - crafty on ICC is running on 4 processors.
>>>>big hardware difference...
>>>>
>>>>it's just what christophe was writing about: crafty is competitive because it
>>>>can use multiple processors (x4 = ~100 rating points). on single processor PCs,
>>>>it is not competitive with top commercials.
>>>>
>>>>the reason is that the commercial programmers write a program for the user who
>>>>buys it - and this user has 1 processor, with *very* few exceptions. if bob had
>>>>spent his time on eval, he would be quite competitive too on single processors
>>>>IMO, but that is not what he chose to do.
>>>>
>>>>i'm speculating here, but probably it's easier for him to focus on
>>>>multiprocessing than on knowledge&search for "political" reasons. when you're
>>>>paid by the university to do research, they want you to do something that the
>>>>people in charge perceive as "useful". tweaking an evaluation function would
>>>>probably sound less useful than accomplishing the parallelization of a complex
>>>>program. parallelization is also interesting for computer science students,
>>>>because the future will bring more multiprocessing systems - also because of
>>>>things like  hyperthreading on one processor.
>>>
>>>
>>>I agree with all your points.  That is why the ICC environment probably reflects
>>>much better the value of each programmer's emphasis, rather than the SSDF list,
>>>which unintentionally favors one programming emphasis over another.
>>
>>but you can't say that it's a coincidence! SSDF tests what 99% of all users
>>have. and that IMO is the right thing to do. a commercial programmer simply
>>can't emphasize multiprocessing to a big extent, because the time spent on that
>>is "wasted" in a commercial sense.
>
>How then do you explain:
>
>deep fritz
>deep shredder
>deep junior
>deep sjeng
>
>for starters.  There are more SMP commercial engines than there are non-SMP
>commercial engines...

i thought some of these could only use 2 processors? not sure about it though. i
also remember reading about absolutely pathetic speedups for dual processors for
some of the commercials.
that's what i mean with "they don't emphasize". perhaps they implement a quick
hack, and call it deep XY. but i don't think any of them has multiprocessing on
the top of the priority list like you do.


>Also care to make a bet about +5 years from now as to whether your statement
>will be true?

no. it should be obvious that the statement is about today. and in fact, it will
still be true then: the statment is "commercial programs will be optimized for
whatever end-users have at home." if that is multiprocessor in 5 years, they
will all have it.


>AMD and Intel will only be selling chips with 2 (or 4) cpus on
>the single chip by then.  Intel is _almost_ doing it now with SMT.  AMD has
>already announced their intent that the next CPU will have two processor cores
>on a single chip.
>
>All the non-SMP guys will then be struggling to catch up...
>
>Those that look ahead architecturally will already "be there".

why are you defending yourself? i didn't suggest that doing SMP was a bad idea.
i just said that it's natural that *you* are doing SMP as a university
professor, because it is an interesting field for academic research (as opposed
to tweaking evaluation functions), and it has practical applications for any
software engineer who wants to learn about using multiple processors for complex
tasks. and the commercials will follow suit once SMP is in everybodys PC.

cheers
  martin




>>and yes, it is great that on ICC you can show up with what you programmed your
>>beast to do.
>>
>>cheers
>>  martin
>>
>>
>>
>>>
>>>
>>>
>>>>
>>>>cheers
>>>>  martin
>>>>
>>>>>ICC Stats for Crafty since March 21, 2004
>>>>>
>>>>>Blitz
>>>>>   Account      win     loss    draw     pctg
>>>>>-- ------------ -----   -----   -----   ------
>>>>>br Deveraux     0       1       0         0.00
>>>>>br SinbadGonnaD 0       3       0         0.00
>>>>>br giant        0       1       1        25.00
>>>>>br glories      0       1       1        25.00
>>>>>br Bitpusher    1       6       9        34.38
>>>>>br BountyHunter 1       3       4        37.50
>>>>>br ajop2        1       1       0        50.00
>>>>>br allAdreamOfA 1       1       1        50.00
>>>>>br bookbuilder  2       2       3        50.00
>>>>>br Joecreek2004 0       0       1        50.00
>>>>>br Lindisfarne  1       1       2        50.00
>>>>>br NubianMagic  0       0       1        50.00
>>>>>br Somnus       1       1       0        50.00
>>>>>br TheBigChill  1       1       1        50.00
>>>>>br Vangard      1       1       0        50.00
>>>>>br pathologist  5       4       3        54.17
>>>>>br X-Engine     12      5       13       61.67
>>>>>br AmazingGrace 19      10      8        62.16
>>>>>br Dhaka        2       1       1        62.50
>>>>>br PostModernis 8       2       4        71.43
>>>>>br stormx       4       0       5        72.22
>>>>>br ajop         2       0       2        75.00
>>>>>br SearcherX    3       0       3        75.00
>>>>>br tlg          4       1       1        75.00
>>>>>br muse-comp    7       1       2        80.00
>>>>>br Amateur      2       0       0       100.00
>>>>>br Clooby       7       0       0       100.00
>>>>>br cro-magnon   1       0       0       100.00
>>>>>br HangerOn     1       0       0       100.00
>>>>>br Nutibara     1       0       0       100.00
>>>>>br rigacombinat 2       0       0       100.00
>>>>>br TAL9000      2       0       0       100.00
>>>>>
>>>>>Standard
>>>>>
>>>>>sr SearcherX    0       1       0         0.00
>>>>>sr Vangard      0       1       0         0.00
>>>>>sr workuta      0       2       1        16.67
>>>>>sr X-Engine     0       1       1        25.00
>>>>>sr DIEP         1       1       0        50.00
>>>>>sr Good-Boy     1       1       1        50.00
>>>>>sr Kronos       0       0       2        50.00
>>>>>sr RuffianY     1       1       1        50.00
>>>>>sr Sukkubus     2       2       4        50.00
>>>>>sr chepla       3       2       4        55.56
>>>>>sr HangerOn     1       0       1        75.00
>>>>>sr stormx       1       0       1        75.00
>>>>>sr thebaron     3       0       1        87.50
>>>>>sr BrassCube    1       0       0       100.00
>>>>>sr SpiderChessX 1       0       0       100.00
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>On a related note, this brings up a question. If it is true that a lot of things
>>>>>>>that give your program improvements at this stage are very minor things, then it
>>>>>>>seems logical that those things would not necessarily result in improvements if
>>>>>>>they were implemented in other programs, because your ideas probably fit into an
>>>>>>>overall system. Do you think it is important to have a good overall system,
>>>>>>>where all components compliment one another?
>>>>>>>
>>>>>>>For instance, a simple example of a system: the job of the full width search is
>>>>>>>to hand off nodes to a qsearch, which has the job of handing off quiet positions
>>>>>>>to an evaluation function. Under that system, you only want to evaluate quiet
>>>>>>>positions, not all positions. If you acheive that, then you make sure your
>>>>>>>qsearch is really delivering quiet positions. If it is, you are probably getting
>>>>>>>accurate analysis from the engine. If someone took that beefed up qsearch that
>>>>>>>was required to make that system work successfully and implemented it in their
>>>>>>>program, it may only cause a qsearch explosion and result in weaker play.
>>>>>>>
>>>>>>>Am I right in believing that it is important to have an overall view of the
>>>>>>>system, and that ideas that resulted in improvements in your engine may not help
>>>>>>>other engines at all?
>>>>>>
>>>>>>
>>>>>>
>>>>>>It is really hard to answer to this question.
>>>>>>
>>>>>>One thing I am convinced of is that if the top chess programmers started to
>>>>>>exchange ideas, like Ed and I did, you would see a significant increase in the
>>>>>>strength of these top programs. Clearly some of them would benefit more.
>>>>>>
>>>>>>
>>>>>>
>>>>>>    Christophe



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.