Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: A New Self-Play Experiment -- Diminishing Returns Shown with 95% Conf.

Author: Tom Kerrigan

Date: 10:05:59 05/25/00

Go up one level in this thread


On May 24, 2000 at 18:03:06, Ernst A. Heinz wrote:

>Hi Dann,
>
>>In these experiments, you varied the depth from 5 to 12 plies, but in the "goes
>>deep" experiments, it was deeper, if I recall correctly.
>
>Yes, right -- but so what?
>
>The "go deep" experiments and the new self-play experiment are not
>really related to one another, or are they?!
>
>>In the earlier experiments, it seemed to level out.
>
>In _which_ experiments did _what_ seem to level out?
>
>>I would like to see control matches where the plies are equal.
>>IOW, 5:5, 6:6 up to 12:12.
>
>As an answer please see the paragraph about "Engine Calibration"
>on page 9.
>
>`` [...] Scaling the engine calibration to other fixed depths
>   requires additional calibration matches at these depths. We did
>   not deem the according effort worthwhile because it is quite
>   unlikely that the Black/White behaviour of "Fritz 6" at other
>   fixed depths differs substantially from the one at 8 plies. ''
>
>>In that way, you will know what the gain of the extra ply is.
>>I don't think you
>>can tell clearly strictly from the data given.
>
>The confidence intervals from Table 6 put bounds on the
>_differences_ in real winning probabilities between versions
>of "Fritz 6" searching to fixed depths "X+1 <=> X".
>
>This very difference exactly quantifies the gain in playing
>strength from depth "X" to "X+1".
>
>>For example, in your 12 <==> 11 experiment, you show 35.71% wins, 52.38% draws,
>>and 11.9% losses.  What would the result be for 12:12?
>
>Does not really matter -- but please see above regarding
>"Engine Calibration".
>
>Which reasons do you have to doubt that "Fritz 6" against
>itself at any same depth "X <=> X" exhibits significantly
>different behaviour than at depth "8 <=> 8"?

But the same question can be asked of your experiment. Why would you believe
that the "8 <=> 9" ply results would vary from the "11 <=> 12" ply results?

-Tom



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.