Author: Ross Boyd
Date: 19:28:17 05/15/03
Go up one level in this thread
On May 15, 2003 at 21:47:20, Jon Dart wrote: >Ernst Heinz did this by using standard test suites, for example Win at Chess, or >ECM (Encyclopedia of Chess Middlegames). He found that the solve rate didn't >really change much with forward pruning on, but the number of nodes searched for >a fixed ply depth decreased 20-50% (this is from the chapter on AEL pruning in >his book Scaleable Search in Computer Chess). He also used other testing >methods, including game play, as detailed in the book. > >--Jon Hi, I use WAC to measure gains in performance but there is a real danger in this. Once as an experiment I modified my qsearch to radically prune 'losing' captures . This resulted in a huge improvement to my WAC scores (from 280 to 292 solved). But my engine played much much worse (than it already did) over the board. So, I concur with Jon (and Ernst, I guess) that gameplay is vital to gauge true performance. I would guess there is no quick way to prove the worth of an algorithm change. Obviously test suites can alert you to obvious problem areas but should be treated with some suspicion. This chess programming thing is a time consuming process, isn't it... :-) Ross > >On May 15, 2003 at 18:17:09, Russell Reagan wrote: > >>I would like to know how to test whether or not a forward pruning method is >>reliable. >> >>I have one idea to test when and if a method is reliable, and I'd like to know >>if it's a good idea or not, and also what other methods might be used to test >>the reliability of forward pruning methods. >> >>My idea requires a collection of games, and two versions of a program. One >>version would have forward pruning turned off, and the other would have it >>turned on. You would feed each version of the program the same game, and let >>each do a search on the initial position to the same fixed depth. If both >>versions report the same move and score, and the version using forward pruning >>had a lower time to depth, then the forward pruning is reliable (so far). If the >>version using forward pruning reported different results, then the forward >>pruning method is not reliable for this type of position. You make the next move >>in the game, and repeat the search and compare the results for each position in >>the game. Then you repeat the process for each game. >> >>When I think about testing the reliability of null-move using this method, I >>think the test would do well. I would expect the test to tell us that in most >>positions, null-move is reliable, and I would expect it to fail for some endgame >>positions, and so this test would tell us that null-move was good forward >>pruning, but to turn it off in the endgame (or detect zugzwang, or however you >>choose to guard against it). I haven't had time to test this though, since I >>just thought of it and I'm not at home. >> >>I am basing all of this on the assumption that the strength forward pruning >>provides is not that it finds better moves at the same depth, but that it >>finishes searching a particular depth in a shorter amount of time, allowing the >>search to go deeper, which is where the added strength comes from. Is this >>correct? >> >>Comments, please...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.