Author: Robert Hyatt
Date: 21:04:03 09/30/04
Go up one level in this thread
On September 29, 2004 at 19:50:15, Stuart Cracraft wrote: >On September 29, 2004 at 17:47:27, martin fierz wrote: > >>On September 29, 2004 at 13:44:50, Stuart Cracraft wrote: >> >>>On September 29, 2004 at 08:22:38, martin fierz wrote: >>> >>>>On September 28, 2004 at 23:49:01, Stuart Cracraft wrote: >>>> >>>>>On September 28, 2004 at 16:29:29, martin fierz wrote: >>>>> >>>>>>On September 28, 2004 at 13:43:48, Stuart Cracraft wrote: >>>>>> >>>>>>>On September 28, 2004 at 08:44:04, martin fierz wrote: >>>>>>> >>>>>>>>On September 28, 2004 at 08:19:15, Stuart Cracraft wrote: >>>>>>>> >>>>>>>>>On September 28, 2004 at 02:14:51, martin fierz wrote: >>>>>>>>> >>>>>>>>>>On September 27, 2004 at 23:45:54, Stuart Cracraft wrote: >>>>>>>>>> >>>>>>>>>>>I experimented with reordering root ply at iterative depth iply > 1 >>>>>>>>>>>where 1 is the root ply, with the results of iply-1 sorted by the >>>>>>>>>>>total nodes of quiescence and main search defined as the # of entries >>>>>>>>>>>for each of those subroutines. >>>>>>>>>>> >>>>>>>>>>>I didn't sort at root node on the first sort by quiescence but instead >>>>>>>>>>>by my normal scheme though I tried quiescence and it was worse. I felt >>>>>>>>>>>this gave a better chance to the above method. >>>>>>>>>>> >>>>>>>>>>>I sorted moves at the root ply for iply > 1 in the following way >>>>>>>>>>>for 7 different parts to the experiment. >>>>>>>>>>> >>>>>>>>>>> sort by normal method (history heuristic, mvv/lva, see, etc. >>>>>>>>>>> sort exactly by subtree node count, nothing else >>>>>>>>>>> sort by subtree node count added to normal score (hh, mvv/lva, see, etc.) >>>>>>>>>>> same as previous but node count x 10 before addition >>>>>>>>>>> same as previous but node count x 100 before addition >>>>>>>>>>> same as previous but node count x 1000 before addition >>>>>>>>>>> same as previous but node count x 10000 before addition >>>>>>>>>>> >>>>>>>>>>>The results, measured by # right on Win-at-Chess varied from >>>>>>>>>>>250 for the first in the list above to 234 for the last. >>>>>>>>>>>Most bunched up between 244-247 except the first was 250, >>>>>>>>>>>my current best on WAC with handtuning everything. >>>>>>>>>>> >>>>>>>>>>>For me, I'm convinced that this style of sorting root ply is >>>>>>>>>>>slightly less good for my short searches compared to what I am using: >>>>>>>>>>>a combination of history, heuristic, see(), and centrality with >>>>>>>>>>>various bonuses, about a half page of code sprinkled about. >>>>>>>>>>> >>>>>>>>>>>The advantage of sorting root node by subtree is the simplicity. >>>>>>>>>>>It eliminates about a half a page of code and introduces >>>>>>>>>>>about a quarter page of code for only slightly lesser results >>>>>>>>>>>(within 1-2% of my current result) so that is good. >>>>>>>>>>> >>>>>>>>>>>Still I think I'll leave it #ifdefed out for now and use it as >>>>>>>>>>>a baseline that is only improvable upon with handtuning of my >>>>>>>>>>>current methods and others to be discovered. >>>>>>>>>>> >>>>>>>>>>>Stuart >>>>>>>>>> >>>>>>>>>>...as ed schröder said to me: "terrible testing". he was right, of course. >>>>>>>>>> >>>>>>>>>>cheers >>>>>>>>>> martin >>>>>>>>> >>>>>>>>>Each to his own. >>>>>>>> >>>>>>>>if you get free advice from one of the world's best computer chess programmers >>>>>>>>it is a good idea to use it. there's not much point writing tons of posts here >>>>>>>>asking for advice if you don't listen.... >>>>>>>> >>>>>>>>cheers >>>>>>>> martin >>>>>>> >>>>>>>Well, condemnations aside, without specific feedback beyond "Oh that's just >>>>>>>bad" (I can get that at work from the boss or from relatives) -- I don't >>>>>>>respond well to that kind of input. It is non-constructive. >>>>>> >>>>>>my post was meant very constructively :-) >>>>>>i just posted something about root move ordering a day or two ago, and ed >>>>>>schröder answered "terrible testing" with a short explanation of why. i expected >>>>>>you had read that thread, and knew what i meant. if not, read it now! >>>>>> >>>>>>cheers >>>>>> martin >>>>>> >>>>>>PS: if you are not in the habit of reading posts of some particular persons >>>>>>(like ed, bob etc) on this board, you should get into that too! other people >>>>>>have something to say too of course, but we do have some >>>>>>world-class-chess-programmers here and i try to read everything they write... >>>>> >>>>>Believe me: I read every character, every sentence, every word, every >>>>>comma, every dot of Ed S. and Bob H. >>>> >>>>good - then i hope you also remember what ed had to say about a similar >>>>experiment of mine! >>>> >>>>cheers >>>> martin >>> >>>Refresh me -- I already spent too much time at this forum!!!! >> >>sure: "terrible testing" :-) >> >>seriously: i changed my root move ordering and reported results in a) 2x40 games >>against other engines and b) in ECMGCP testsuite. ed made the statement i quoted >>above. >> >>he says: if you only change move ordering, you should keep a testset handy which >>you search to fixed depth, and where you can compare how move ordering affects >>the node count on your test set. best not to compare total nodes, because one >>bad position can skew your results, take e.g. #of positions with less nodes vs >>#of positions with more nodes; or take the geometric mean over the test set of >>newnodes/oldnodes. >> >>this is much a better test of move ordering, because it does not rely on your >>engine accidentally finding or not finding a certain move. >> >>that is what ed says, and i believe he is right :-) >> >>hmm, i could have written this to start with and spared us a lot of >>ping-pong-posts ;-) >> >>cheers >> martin > >Yes -- I remember Ed's comments -- I plan to implement it. Another >one is Bob's idea to complete 1 ply searches but let the current ply >finish before returning. > >Good stuff. > >Stuart That will help a lot, but it won't cure the problem as Martin pointed out. You still have enough timing variance to barely start the next iteration one time but not the next, which will make the results incomparable... But if you are making decisions that affect the _size_ of the tree (ie different root move ordering, or search extensions) then you have to actually _measure_ the size of the tree, not something else. Using a timed search is measuring something else entirely, not the thing directly influenced by your changes...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.