Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Test your program

Author: Robert Hyatt

Date: 08:00:25 05/05/01

Go up one level in this thread


On May 05, 2001 at 09:40:01, Ralf Elvsén wrote:

>On May 05, 2001 at 08:46:52, Jesper Antonsson wrote:
>
>>On May 05, 2001 at 00:53:39, Robert Hyatt wrote:
>>>On May 05, 2001 at 00:20:16, Peter Kappler wrote:
>>>>>OK... then at _today's_ computer speeds, I don't believe in diminishing
>>>>>returns yet.  In 20 years, perhaps.  But the difference between a 15 ply
>>>>>search and a 17 ply search is _significant_ still.  Lots of experiments have
>>>>>shown that diminishing returns don't appear to happen at any depth we can
>>>>>reach today, even using 24 hours of computer time.
>>>>
>>>>What about Ernst Heinz's fixed-depth, self-play matches with Fritz? They
>>>>seemed to strongly suggest diminishing returns, even at depths much
>>>>shallower than 15 or 17 plies.
>>>>
>>>Perhaps the program?  Hans Berliner did an interesting experiment a long while
>>>back, and concluded that "dumber" programs show this diminishing return problem
>>>sooner than "smarter" programs.  Ernst also concluded that for the time being,
>>>at least thru 15-16 plies, there was no apparent 'diminishing returns' for his
>>>program when he replicated the tests Monty and I did...
>>>
>>>I don't say there is no diminishing return.  I say I don't see any real
>>>evidence to support the idea just yet....
>>
>>I disagree. In Ernst Heinz's experiment "Dark Though goes Deep"
>><http://supertech.lcs.mit.edu/~heinz/dt/node46.html>, and in a similar
>>experiment before his that you did with Crafty, the rate of best-move changes
>>from one ply to another clearly went down as depth went up. The margin of error
>>is a bit high to draw any real conclusions from the changes at the greatest
>>depths, but the trend is clear nonetheless.
>>
>>Furthermore, I think that experimental data is not really needed, diminishing
>>returns in this sense (in a rating sense, I have no idea, however) must exist.
>>The deeper you go, the more best moves will be found for the right reasons (and
>>the more inferior moves will be discarded), and after that the best move
>>returned won't change (as much).
>>
>>When I fit an exponential curve to Heinz's results (and extrapolate), I get
>>approximately these best change rates:
>>
>>1
>>2	37,5%
>>3	34,6%
>>4	31,9%
>>5	29,4%
>>6	27,1%
>>7	25,0%
>>8	23,0%
>>9	21,2%
>>10	19,5%
>>11	18,0%
>>12	16,6%
>>13	15,3%
>>14	14,1%
>>15	13,0%
>>16	12,0%
>>17	11,0%
>>18	10,2%
>>19	9,4%
>>20	8,6%
>
>Bold extrapolation... :)
>
>>
>>This means that going from ply 9 to 10 gives about as much as going from ply 17
>>to 19. The returns are still great on the depths where programs usually play
>>today and the returns taper off very slowly, but I'm convinced they *do* taper
>>off.
>>
>>Jesper
>
>I have yet to see a convincing argument why the rate of best-move changes
>would be so directly related to playing strength.

If you believe that another ply gives a more accurate answer, which I do,
then the rate of change should be obvious.  If you change your mind, you
find a better move due to the deeper depth.




>
>I think Ernst's self play experiment with Fritz is the one to look at since
>he addresses the immediate question, and
>he thought it proved diminishing returns to a certain degree. That one or
>two extra plies gives a benefit is of course true, the question is how much.
>
>Ralf



This page took 0.03 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.