Author: Christophe Theron
Date: 16:14:22 04/14/04
Go up one level in this thread
On April 14, 2004 at 10:15:37, Matthew Hull wrote:
>On April 14, 2004 at 02:51:29, Christophe Theron wrote:
>
>>On April 13, 2004 at 17:00:24, Matthew Hull wrote:
>>
>>>On April 13, 2004 at 14:21:07, Christophe Theron wrote:
>>>
>>>>On April 13, 2004 at 01:29:02, Russell Reagan wrote:
>>>>
>>>>>On April 12, 2004 at 23:07:46, Christophe Theron wrote:
>>>>>
>>>>>>Further, wouldn't you just *hate* if I took the fun out of chess programming by
>>>>>>telling you everything? :)
>>>>>
>>>>>My gut feeling is that we would probably be disappointed for the most part. I
>>>>>bet a lot of us think all of you commercial authors are harboring lots of
>>>>>magical secrets that can turn an average program into a beast. Something similar
>>>>>to the improvements you get by going from minimax to alphabeta, or by adding
>>>>>null-move to an average program, and things like that. Those are very
>>>>>significant improvements.
>>>>>
>>>>>I have received the impression from you and other sources like Ed's webpage that
>>>>>this is not the case. There are some clever things on Ed's webpage, but for the
>>>>>most part, it is good ideas based on common sense, and then taking the time and
>>>>>effort to hammer out every last detail to make an idea work, followed by an
>>>>>efficient implementation.
>>>>>
>>>>>To illistrate the difference between what I think a lot of people would expect
>>>>>to hear from you if you divulged all of your secrets and what I think we would
>>>>>really get, consider null-move. Null-move is something that you can add to a
>>>>>program that uses no forward pruning, and once you spend a small amount of time
>>>>>getting it to work right, the program suddenly plays like it's on steroids
>>>>>(relatively speaking). However, if we took an average program and added in a few
>>>>>ideas from Ed's webpage, I wouldn't expect nearly as big of an improvement. I
>>>>>think you guys just take a lot of ideas and get small improvements here and
>>>>>there, and at the end of the decade, it amounts to a big improvement. 10%
>>>>>reduction in tree size here, 20% there, it adds up.
>>>>>
>>>>>Am I right? If we are expecting to see magical earth shattering secrets, would
>>>>>we be disappointed?
>>>>
>>>>
>>>>
>>>>I don't think you would be disappointed.
>>>>
>>>>But you are right in assuming that you would not see a dramatic improvement such
>>>>as the one you get from alpha-beta vs minimax.
>>>>
>>>>You know, one has to wonder where the difference in elo strength between Crafty
>>>>and the top commercial comes from.
>>>
>>>
>>>Compare this with your mileage at home. Many of the plus performance scores are
>>>against accounts running commercial programs.
>>>
>>>ICC Stats for Crafty since March 21, 2004
>>
>>
>>
>>Stats on chess servers are extremely unreliable.
>>
>>This is as close to a scientific experiment as we will ever get:
>>http://w1.859.telia.com/~u85924109/ssdf/list.htm
>
>
>This is really a uniform platform experiment which does not reflect the Crafty
>value proposition. The ICC environment ensures that Bob's program is setup
>optimally and also utilizes those strong points in which he has invested his
>time. So the real value of the project is better reflected in the latest CCT.
>
>There are 16-way and 32-way machines that Crafty can utilize as a result of it's
>development emphasis. Commercial offereings cannot match this (AFAIK).
>Therefore, in an open-hardware competition, the Crafty project must be seen as
>the project to beat.
>
>Another scary thought is that Bob is clearly aware of other known improvements
>which he has not had time to implement like advanced foreward pruning, eval
>tweaks/improvements, pawn-fortress code, etc.
>
>SSDF running Crafty is kind of like taking the Cray Blitz fortran code source
>and compiling it for a little uni-processor intel box, and then wondering why it
>cannot compete with Shredder. Crafty is designed for open hardware. limiting
>it to small hardware for strength comparison is missing the point of the
>project, IMO.
The problem with chess servers is the very high level of noise. I wouldn't use
them to measure the strength of any program.
I also do not see why Crafty would be exempted of measuring its strength on a
standard PC. I am not saying that it would not perform better on open hardware,
but I also think that it is interesting to assess its strength on a more
standardized hardware.
My point is not to belittle the merits of the Crafty project here. It's just
that, because it is open source, it offers a way to measure the strength
difference between a state of the art open source chess program (using public
domain chess algorithms) and state of the art closed source chess programs.
Christophe
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.