Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: A New Self-Play Experiment -- Diminishing Returns Shown with 95% Conf.

Author: Ratko V Tomic

Date: 13:42:47 05/25/00

Go up one level in this thread


> I can't think of any reason why the result of X:X matches (between
> the same engine) would vary with depth.
>
> What could possibly cause this?

The same thing which causes X+1 to win more against X i.e. the
existence of shallow terminal (or practically terminal) nodes.
What test X:X for X=7,8,9... would assure is that the sample
set of positions is fair in distributing such shallow terminal
nodes.

One can illustrate the problem with an extremely skewed sample:
suppose that all positions have a tempting trap with a reward,
say, 9 plies ahead. Your sample fairness test 8:8 would not show
the problem here, yet the 9:8 play would allow 9 ply program
to win every game (with right colors) against the 8 ply program
which would always fall for the trap.

While the positions you used are likely not anywhere near as skewed
in this regard, it is conceivable that a few percent variation may
occur in X:X play with variable X (your data does show some
non-uniformity of X+1 effects with depth). So the result X:X+1
should be normalized to X:X and X+1:X+1 results and not to 8:8
results for all X. Or the positions should be prefiltered to those
yielding approximately 50:50 for all X:X matches.

In any case, the results are still quite interesting and useful. One
possible future test not mentioned in the preprint would be to try
X+Y:X, where Y can vary as 1,2,3.... Such test would provide the
idea of degree of stability (across the strength differences) of the
diminishing returns effect.




This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.