Author: Sune Fischer
Date: 17:31:32 02/06/02
Go up one level in this thread
On February 06, 2002 at 20:03:10, Odd Gunnar Malin wrote: >On February 06, 2002 at 18:03:41, Sune Fischer wrote: > >>On February 06, 2002 at 17:43:09, Dann Corbit wrote: >> >>>On February 06, 2002 at 15:35:28, Sune Fischer wrote: >>>[snip] >>>>Well I don't disagree with that, but suppose in any position we knew what the >>>>best move(s) was. Now how often would a n-ply search find the correct move(s)? >>>>This will give some kind of probability distribution on the plies. I'm sure this >>>>is directly related to the diminishing returns at the higher plies. >>> >>>Diminishing returns at higher plies has not been demonstrated. The research by >>>both Heinz and Hyatt is statistically inconclusive. >>> >>>If you make a program with a material only eval, it will swim through plies like >>>a salmon on steroids. And it will lose all of its chess games to programs >>>searching half as deep. >>> >>>IOW -- plies don't tell the whole story. And what happens when we gain another >>>ply is still somewhat mysterious as far as what the value should be. >> >>Pure speculation on my part here, but if the program did a brute force search >>to ply n (no extensions), using only material evaluation, then it should be >>possible to prove diminishing returns(?) (see below). >> >> >>>Here is a puzzler... >>>When we search another ply, we typically expend more effort than all the >>>previous plies combined. Why is it (then) that instead of being at least twice >>>as good in the evaluation, we are only fractionally better? After all, we are >>>looking at exponentially more board positions. It's a bit odd that our strength >>>does not increase at least linearly (or does it, at some point?) >>> >>>The strength increase looks a bit like a decaying exponential (considering the >>>graphs from available research papers and ignoring the enormous error bars). >> >>Yes, that is actually what I've been trying to explain. >>I think I understand the nature of that decaying/descending sequence. >> >>Suppose you have millions of given random test positions in which you *know* the >>best move(s). >>Now run tests to see how often a 1-ply search will find the correct move, and >>how often a 2-ply search will find the correct move etc. >>Line up all these percentiles, and you will probably get something like this: >> >>1-ply search: 40% correct moves >>2-ply ......: 55% correct moves >>3-ply ......: 65% correct moves >>4-ply ......: 72% correct moves >>etc... >> >>The thing is, that the percentiles _must_ converge towards 100, so it will need >>to slow down, there may only be 2% difference between a 12 and 13 ply search, >>which is why it is really hard to measure anything. >> >>-S. > >This would tell you that a 10 ply search is better than a 9 ply search but how >should you calculate the strenght out of this? I don't know about the rating, I was merely interested in the problem of diminished returns. The connection to rating is there, somewhere. >Correct moves is a linear scala but why should the strength difference between >20% and 30% be equal to the strength difference between 80% and 90%. Naturally I can't answer to your question, only make a guess: if program A picks the right move 10% more often than program B, then I think _it would_ be equivalent to a constant rating difference, no matter the rating of A and B. I cannot prove it, but it could be tested:) >With this selfplay test we already have a scale for strength (rating). Yes, I am also surprized it can't be used to *prove* diminishing returns. The problem with this method is perhaps that the distribution is too flat. >Odd Gunnar -S.
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.