Author: Tom Likens
Date: 11:00:13 05/25/04
Go up one level in this thread
On May 24, 2004 at 14:39:36, Ed Schröder wrote: >On May 24, 2004 at 13:09:42, José Carlos wrote: > >>On May 24, 2004 at 12:39:35, Ed Schröder wrote: >> >>>On May 24, 2004 at 12:07:56, Russell Reagan wrote: >>> >>>>On May 24, 2004 at 10:13:33, Ed Schröder wrote: >>>> >>>>>>It would also be interesting to hear some background - how you arrived at what, >>>>>>what else you tried & discarded, how confident you are about it, any general >>>>>>thoughts, etc. >>>>> >>>>>That would take too much to read, I prefer to stay on topic and as short as >>>>>possible. Your question is much better suited for a forum, so: I feel my way of >>>>>doing king safety is okay, I can't say that about the evaluation of passed >>>>>pawns, I feel most insecure about it. >>> >>>>It would be interesting to know how you determine if a change you make is an >>>>improvement (if you are allowed to tell us, of course). >>> >>>You guessed well, I can't tell. >>> >>>Ed > >> But Ed, you surely had a method to determine that way before meeting >>Christophe (which is the reason, I guess, you can't tell now). > >Yes. > > >>Could you comment >>on your pre-Christophe testing methods? >> Thanks in advance. > >I can tell you one part, here is. I have testsets that contain well selected >positions (tactics, positional, midgame, endgame etc.) Depending on the changes >I have made I run a specific testset (or testsets) as first step in my own >text-based DOS interface, it serves as a first smell. When finished I can >produce several reports, such as to compare the results with a previous version. > >A screenshot: > >Rebel ... version RXP 05-07-2003 >(REBEL XP) (A-2000) CV=100 > >Rebel ... version M02 04-04-2004 >(KILLER - KD1|KD2|KD1-2|KD2-2) CV=RXP > >Number of searched positions : 260 >Number of different moves : 5 >Number of different scores : 8 (Interval score = 0.10) >Number of different times : 22 (Interval time = 10%) > >Total time (version 1) : 00:57:56 Nodes 3.187.570 >Total time (version 2) : 00:57:32 Nodes 3.151.571 >Total percent : 1% (1.14%) > >Score 0.01 - 0.10 22 Nodes faster vs slower 119 - 117 >Score 0.10 - 0.25 5 Nodes (1%) 119 - 51 >Score 0.25 - 0.50 1 Nodes (2%) 61 - 34 >Score 0.50 - 0.75 0 Nodes (3%) 42 - 26 >Score 0.75 - 1.00 1 Nodes (4%) 36 - 20 >Score 1.00 - 2.00 0 Nodes (5%) 31 - 19 >Score greater 2.00 1 Nodes (9%) 12 - 13 > >Compare on <A>ll <M>ove <S>core <T>ime <N>odes <Q>uit > >================= Ed, Thanks for posting this but I want to make sure I understand what you've written. Specifically, I have a question about the "Nodes faster vs slower" section. Does the line: Nodes (1%) 119 - 51 indicate that version 2 had 119 positions where it searched 1% fewer nodes than version 1 *and* 51 positions where it searched 1% more nodes (or am I missing the point altogether)? Also regarding the two nodes that ran much longer, did they also search a proportionate number of extra nodes. I'm wondering because I would have expected them to show up in the "Nodes-Overview" section. And finally, I'm also wondering if it would be advantageous to have a "Time-Overview" section analogous to the score and nodes data, (actually, it sounds like you may already have this in the other analysis funtion you mentioned). regards, --tom > >In this specific case (it is just a random example) I have tested a change in >move-ordering swapping the order of killer-moves to see if it gains anything. > >The report says that 260 positions have been searched, that the change produces >5 different moves, 8 different scores outside an interval of 0.10, 22 times >outside an interval of of 10%. Of course intervals are flexible. > >Then the report shows that the change is good for a 1% speedup (ahem) in time as >well as nodes. The "Nodes-Overview" (faster vs slower) reveals that there are >more positions that run faster than slower which in this specific case is an >important piece of information because the speed gain is so low and from another >analysis function I could see the 1% was polluted by 2 positions that ran much >longer and by skipping them the speed gain was higher. The consistency as shown >in the nodes-overview becomes a dominant factor to accept a change (or not) in >doubtful cases like this one. > >Another function (which I think every serious chess programmer should have) is >the "Nodes-compare" function. It will report every difference in nodes of the >tested positions, 260 in this case. It has been extremely useful for me to avoid >bugs. > >Ed
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.