Computer Chess Club Archives




Subject: Re: PB-ON vs PB-OFF (final results)

Author: Ratko V Tomic

Date: 08:55:16 10/16/99

Go up one level in this thread

> Even after seeing your equations, I'm still not sure how you can
> determine the pondering percentage by the winning percentage.

If you know the rating difference you can compute total/effective thinking time
difference or time ratio (using D=C*(R-1), using value C=100 rating points gain
for doubling the thinking time). Once you have full thinking times ratio, and
you know the time controls each _nominally_ got, you will notice that the
effective thinking time ratio (as obtained from rating difference) is greater
than the nominal time controls ratio, e.g. in the 1st match both had equal
nominal times, but the time ratio from performance difference was 1.78, i.e. the
pondering Rebel had more actual/effective time. Hence you can compute how much
extra time the pondering version got to reach the times ratio expected from the
performance ratings difference. That extra time can only come from the pondering
 time gain, and that's how you get G, the gain hit rate. (Obviously, in a fairly
small sample as this you have a significant statistical error marging, so take
the numbers 80% with a grain of salt.)

>If pondering Rebel had win 100% of the games, would you say it pondered
>with 100% accuracy?

No. That could be only due to a statistical fluctuation in a 100 (or any finite
number) game match. For example, if instead of different time controls you used
depth D and D+1 versions, the D+1 will guess correctly the move of D version in
every single move, 100% guess rate. Yet it will lose 20-30 percent of games.
Guessing what the opponent will play doesn't mean it will find a correct answer
to that move.

>(And were you not the one saying that a shallower
>search can produce ultimately better moves in
>many cases? In these cases, a correct pondering would _worsen_ the result. :)

It's not something I made up. There are papers with D vs D+1 types of results.
So it is a well known fact, not an opinion. A shallower search can in some "good
 percentage" of positions produce better move than the deeper search. Obviously
the exact meaning of "good percentage" depends on positions, on depths used for
the two versions and the kind of search algorithms and evaluations used. In any
case, it is not anywhere near zero or negligible.

The cause of this seemingly paradoxical phenomenon are the inaccuracies in the
evaluation functions and the tree traversal approximations used in minimaxing
(which cut down the branching factor below sqrt(Width) which is the rigourous
minimaxing optimum i.e. from 6 down to 3 or 4 in a typical middle game; e.g.
some kind of forward pruning or other heuristic shortcuts).

So when program evaluates some root move, and comes back with value V, that
means only a kind of median or average value of a distribution of values, as if
you said that some nation has $9500 per capita on the bank accounts. Some people
will have more, some less, that's only the average. So when program comes back
with Value(Move)=1.5 pawns, that's a hypothetical average (its estimate) over
unknown distribution of values (which can be sampled over many games). In each
individual position the actual value (as a perfect player would give) will be
almost always completely different, just as your savings account will be almost
always different from the national per capita savings average.

Therefore, the phenomenon of a shallower search finding a better move than a
deeper search, is as "strange" as someone from US having a larger saving account
than someone from Japan, even though per capita savings for Japanese may be
larger than for Americans.

>Again, I thank Ed for running this experiment, and hope that he or others can
>continue to make similar experiments to determine The Truth. :)

Yes, the Ed's experiment was a nice result to have.

This page took 0.07 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.