Author: Stephen A. Boak
Date: 08:17:15 04/28/01
Go up one level in this thread
On April 28, 2001 at 06:03:46, Uri Blass wrote: >On April 28, 2001 at 05:00:22, Dan Andersson wrote: > >>I remember that Ernst A. Heinz found diminishing returns. There are diminishing >>returns in many other two-player games e.g. Reversi and Checkers. One possible >>explanation for its absence is that chess evaluation functions are of a low >>quality, because that chess is more complex. >> >>Regards Dan Andersson > >I chose the nunn position and not regular games. >There is a reason for it. > >I believe that theory sometimes leads to relatively simple drawn positions if >you search deep enough and it may explain diminishing returns when I believe >that the nunn positions are more complicated and you may need more plies to get >the draw when depth is not important so it is going to be more hard to prove >diminishing returns on the nuun2 match. > >I believe that if I continue this nunn2 experiment to Deep Fritz depthes 3-15 >and tiger14 depth 3-15 then diminishing returns will be demonstrated >statistically. > >I am going to do it at least for depth 9-10 and I did not decide how much I am >going to continue it(one reason is that the exact computer time that I need is >not clear to me) > >Matches at depth 8 or even 9 can take a short time but matches at bigger >depthes are going to take more time. > >I guess that the time that is needed after upgrading my hardware is something >like 3^(d-9) hours per match at fixed depth d against smaller depth. >and it means 27 hours per match at depth 12,243 hours per match at depth 14, >729 hours per match at depth 15. > >If I play at least 11 matches at depth 15 in order to get 300 games for >tiger(depth 15) and Fritz (depth 15) then I need something like 7290 hours that >is >almost a year of computer time. > >I am not going to give my computer more than an average of few hours in a day >so I need some years to complete the experiment and it may be faster to wait for >a faster hardware before starting matches at big depthes. > >Uri The problem is probably not the time per match. It is the huge quantity of matches that must be played to observe any score other than 100% score for the higher ply program and 0% score for the lower ply program, when the delta ply is large. Don't forget, at ply delta of only 6, you already have a 50-0 score. The relative Elo formula doesn't work for such a score. Maybe after a few hundred games, the score won't be 100% to 0%, for ply delta of 6. But how bad will the situation be with a ply delta of 8, 10, or 12? A bell-shaped curve isn't guaranteed to work at the extreme legs. It would be foolhardy to rely on it working well in such cases. However, to extrapolate to a ply delta of, say 10, worse yet 15, how many hundreds of millions of games will you have to play to not have a score of, say, 300,000,000 - 0. Even if future hardware plays such a match in 1 minute, you will likely be a bit short in longevity to see a meaningful score other than 100% - 0%. I am exaggerating to some degree. If you play this many games, I believe all programs will show a 'bug' somewhere along the line, and lose or draw a game or two. But, do you honestly think that a program searching to delta ply depth 15 (N+15) will ever 'normally' draw or lose to a program searching to only ply N? The amount of bugs in a program may be at the 1 in a million positions level. However, if it already is searching 15 ply deeper than its opponent, it will have 14 more plies to 'get rid of the bug' by seeing past it, or at least to still outplay its opponent by seeing more accurately. Chances are, the lower ply depth searcher will make a fatal mistake, before the higher ply searcher gets caught in a 'bug'. Perhaps the ultimate limiting factor, that might lead to diminishing returns prior to the endgame, is the relative level of bugs in two competing programs! --Steve
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.