Author: Stuart Cracraft
Date: 16:40:18 10/01/04
Go up one level in this thread
On October 01, 2004 at 00:04:03, Robert Hyatt wrote: >On September 29, 2004 at 19:50:15, Stuart Cracraft wrote: > >>On September 29, 2004 at 17:47:27, martin fierz wrote: >> >>>On September 29, 2004 at 13:44:50, Stuart Cracraft wrote: >>> >>>>On September 29, 2004 at 08:22:38, martin fierz wrote: >>>> >>>>>On September 28, 2004 at 23:49:01, Stuart Cracraft wrote: >>>>> >>>>>>On September 28, 2004 at 16:29:29, martin fierz wrote: >>>>>> >>>>>>>On September 28, 2004 at 13:43:48, Stuart Cracraft wrote: >>>>>>> >>>>>>>>On September 28, 2004 at 08:44:04, martin fierz wrote: >>>>>>>> >>>>>>>>>On September 28, 2004 at 08:19:15, Stuart Cracraft wrote: >>>>>>>>> >>>>>>>>>>On September 28, 2004 at 02:14:51, martin fierz wrote: >>>>>>>>>> >>>>>>>>>>>On September 27, 2004 at 23:45:54, Stuart Cracraft wrote: >>>>>>>>>>> >>>>>>>>>>>>I experimented with reordering root ply at iterative depth iply > 1 >>>>>>>>>>>>where 1 is the root ply, with the results of iply-1 sorted by the >>>>>>>>>>>>total nodes of quiescence and main search defined as the # of entries >>>>>>>>>>>>for each of those subroutines. >>>>>>>>>>>> >>>>>>>>>>>>I didn't sort at root node on the first sort by quiescence but instead >>>>>>>>>>>>by my normal scheme though I tried quiescence and it was worse. I felt >>>>>>>>>>>>this gave a better chance to the above method. >>>>>>>>>>>> >>>>>>>>>>>>I sorted moves at the root ply for iply > 1 in the following way >>>>>>>>>>>>for 7 different parts to the experiment. >>>>>>>>>>>> >>>>>>>>>>>> sort by normal method (history heuristic, mvv/lva, see, etc. >>>>>>>>>>>> sort exactly by subtree node count, nothing else >>>>>>>>>>>> sort by subtree node count added to normal score (hh, mvv/lva, see, etc.) >>>>>>>>>>>> same as previous but node count x 10 before addition >>>>>>>>>>>> same as previous but node count x 100 before addition >>>>>>>>>>>> same as previous but node count x 1000 before addition >>>>>>>>>>>> same as previous but node count x 10000 before addition >>>>>>>>>>>> >>>>>>>>>>>>The results, measured by # right on Win-at-Chess varied from >>>>>>>>>>>>250 for the first in the list above to 234 for the last. >>>>>>>>>>>>Most bunched up between 244-247 except the first was 250, >>>>>>>>>>>>my current best on WAC with handtuning everything. >>>>>>>>>>>> >>>>>>>>>>>>For me, I'm convinced that this style of sorting root ply is >>>>>>>>>>>>slightly less good for my short searches compared to what I am using: >>>>>>>>>>>>a combination of history, heuristic, see(), and centrality with >>>>>>>>>>>>various bonuses, about a half page of code sprinkled about. >>>>>>>>>>>> >>>>>>>>>>>>The advantage of sorting root node by subtree is the simplicity. >>>>>>>>>>>>It eliminates about a half a page of code and introduces >>>>>>>>>>>>about a quarter page of code for only slightly lesser results >>>>>>>>>>>>(within 1-2% of my current result) so that is good. >>>>>>>>>>>> >>>>>>>>>>>>Still I think I'll leave it #ifdefed out for now and use it as >>>>>>>>>>>>a baseline that is only improvable upon with handtuning of my >>>>>>>>>>>>current methods and others to be discovered. >>>>>>>>>>>> >>>>>>>>>>>>Stuart >>>>>>>>>>> >>>>>>>>>>>...as ed schröder said to me: "terrible testing". he was right, of course. >>>>>>>>>>> >>>>>>>>>>>cheers >>>>>>>>>>> martin >>>>>>>>>> >>>>>>>>>>Each to his own. >>>>>>>>> >>>>>>>>>if you get free advice from one of the world's best computer chess programmers >>>>>>>>>it is a good idea to use it. there's not much point writing tons of posts here >>>>>>>>>asking for advice if you don't listen.... >>>>>>>>> >>>>>>>>>cheers >>>>>>>>> martin >>>>>>>> >>>>>>>>Well, condemnations aside, without specific feedback beyond "Oh that's just >>>>>>>>bad" (I can get that at work from the boss or from relatives) -- I don't >>>>>>>>respond well to that kind of input. It is non-constructive. >>>>>>> >>>>>>>my post was meant very constructively :-) >>>>>>>i just posted something about root move ordering a day or two ago, and ed >>>>>>>schröder answered "terrible testing" with a short explanation of why. i expected >>>>>>>you had read that thread, and knew what i meant. if not, read it now! >>>>>>> >>>>>>>cheers >>>>>>> martin >>>>>>> >>>>>>>PS: if you are not in the habit of reading posts of some particular persons >>>>>>>(like ed, bob etc) on this board, you should get into that too! other people >>>>>>>have something to say too of course, but we do have some >>>>>>>world-class-chess-programmers here and i try to read everything they write... >>>>>> >>>>>>Believe me: I read every character, every sentence, every word, every >>>>>>comma, every dot of Ed S. and Bob H. >>>>> >>>>>good - then i hope you also remember what ed had to say about a similar >>>>>experiment of mine! >>>>> >>>>>cheers >>>>> martin >>>> >>>>Refresh me -- I already spent too much time at this forum!!!! >>> >>>sure: "terrible testing" :-) >>> >>>seriously: i changed my root move ordering and reported results in a) 2x40 games >>>against other engines and b) in ECMGCP testsuite. ed made the statement i quoted >>>above. >>> >>>he says: if you only change move ordering, you should keep a testset handy which >>>you search to fixed depth, and where you can compare how move ordering affects >>>the node count on your test set. best not to compare total nodes, because one >>>bad position can skew your results, take e.g. #of positions with less nodes vs >>>#of positions with more nodes; or take the geometric mean over the test set of >>>newnodes/oldnodes. >>> >>>this is much a better test of move ordering, because it does not rely on your >>>engine accidentally finding or not finding a certain move. >>> >>>that is what ed says, and i believe he is right :-) >>> >>>hmm, i could have written this to start with and spared us a lot of >>>ping-pong-posts ;-) >>> >>>cheers >>> martin >> >>Yes -- I remember Ed's comments -- I plan to implement it. Another >>one is Bob's idea to complete 1 ply searches but let the current ply >>finish before returning. >> >>Good stuff. >> >>Stuart > > >That will help a lot, but it won't cure the problem as Martin pointed out. You >still have enough timing variance to barely start the next iteration one time >but not the next, which will make the results incomparable... > >But if you are making decisions that affect the _size_ of the tree (ie different >root move ordering, or search extensions) then you have to actually _measure_ >the size of the tree, not something else. Using a timed search is measuring >something else entirely, not the thing directly influenced by your changes... Absolutely -- but I think either that or bumping up from 1 second per move to a few seconds per move, or simply buying the fastest machine out there now would do take care of the concerns.
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.