Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Test your program

Author: Jesper Antonsson

Date: 05:46:52 05/05/01

Go up one level in this thread


On May 05, 2001 at 00:53:39, Robert Hyatt wrote:
>On May 05, 2001 at 00:20:16, Peter Kappler wrote:
>>>OK... then at _today's_ computer speeds, I don't believe in diminishing
>>>returns yet.  In 20 years, perhaps.  But the difference between a 15 ply
>>>search and a 17 ply search is _significant_ still.  Lots of experiments have
>>>shown that diminishing returns don't appear to happen at any depth we can
>>>reach today, even using 24 hours of computer time.
>>
>>What about Ernst Heinz's fixed-depth, self-play matches with Fritz? They
>>seemed to strongly suggest diminishing returns, even at depths much
>>shallower than 15 or 17 plies.
>>
>Perhaps the program?  Hans Berliner did an interesting experiment a long while
>back, and concluded that "dumber" programs show this diminishing return problem
>sooner than "smarter" programs.  Ernst also concluded that for the time being,
>at least thru 15-16 plies, there was no apparent 'diminishing returns' for his
>program when he replicated the tests Monty and I did...
>
>I don't say there is no diminishing return.  I say I don't see any real
>evidence to support the idea just yet....

I disagree. In Ernst Heinz's experiment "Dark Though goes Deep"
<http://supertech.lcs.mit.edu/~heinz/dt/node46.html>, and in a similar
experiment before his that you did with Crafty, the rate of best-move changes
from one ply to another clearly went down as depth went up. The margin of error
is a bit high to draw any real conclusions from the changes at the greatest
depths, but the trend is clear nonetheless.

Furthermore, I think that experimental data is not really needed, diminishing
returns in this sense (in a rating sense, I have no idea, however) must exist.
The deeper you go, the more best moves will be found for the right reasons (and
the more inferior moves will be discarded), and after that the best move
returned won't change (as much).

When I fit an exponential curve to Heinz's results (and extrapolate), I get
approximately these best change rates:

1
2	37,5%
3	34,6%
4	31,9%
5	29,4%
6	27,1%
7	25,0%
8	23,0%
9	21,2%
10	19,5%
11	18,0%
12	16,6%
13	15,3%
14	14,1%
15	13,0%
16	12,0%
17	11,0%
18	10,2%
19	9,4%
20	8,6%

This means that going from ply 9 to 10 gives about as much as going from ply 17
to 19. The returns are still great on the depths where programs usually play
today and the returns taper off very slowly, but I'm convinced they *do* taper
off.

Jesper



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.