Author: Ed Schröder
Date: 14:56:15 05/24/04
Go up one level in this thread
On May 24, 2004 at 16:15:53, Peter Fendrich wrote: >On May 24, 2004 at 14:39:36, Ed Schröder wrote: > >>On May 24, 2004 at 13:09:42, José Carlos wrote: >> >>>On May 24, 2004 at 12:39:35, Ed Schröder wrote: >>> >>>>On May 24, 2004 at 12:07:56, Russell Reagan wrote: >>>> >>>>>On May 24, 2004 at 10:13:33, Ed Schröder wrote: >>>>> >>>>>>>It would also be interesting to hear some background - how you arrived at what, >>>>>>>what else you tried & discarded, how confident you are about it, any general >>>>>>>thoughts, etc. >>>>>> >>>>>>That would take too much to read, I prefer to stay on topic and as short as >>>>>>possible. Your question is much better suited for a forum, so: I feel my way of >>>>>>doing king safety is okay, I can't say that about the evaluation of passed >>>>>>pawns, I feel most insecure about it. >>>> >>>>>It would be interesting to know how you determine if a change you make is an >>>>>improvement (if you are allowed to tell us, of course). >>>> >>>>You guessed well, I can't tell. >>>> >>>>Ed >> >>> But Ed, you surely had a method to determine that way before meeting >>>Christophe (which is the reason, I guess, you can't tell now). >> >>Yes. >> >> >>>Could you comment >>>on your pre-Christophe testing methods? >>> Thanks in advance. >> >>I can tell you one part, here is. I have testsets that contain well selected >>positions (tactics, positional, midgame, endgame etc.) Depending on the changes >>I have made I run a specific testset (or testsets) as first step in my own >>text-based DOS interface, it serves as a first smell. When finished I can >>produce several reports, such as to compare the results with a previous version. >> >>A screenshot: >> >>Rebel ... version RXP 05-07-2003 >>(REBEL XP) (A-2000) CV=100 >> >>Rebel ... version M02 04-04-2004 >>(KILLER - KD1|KD2|KD1-2|KD2-2) CV=RXP >> >>Number of searched positions : 260 >>Number of different moves : 5 >>Number of different scores : 8 (Interval score = 0.10) >>Number of different times : 22 (Interval time = 10%) >> >>Total time (version 1) : 00:57:56 Nodes 3.187.570 >>Total time (version 2) : 00:57:32 Nodes 3.151.571 >>Total percent : 1% (1.14%) >> >>Score 0.01 - 0.10 22 Nodes faster vs slower 119 - 117 >>Score 0.10 - 0.25 5 Nodes (1%) 119 - 51 >>Score 0.25 - 0.50 1 Nodes (2%) 61 - 34 >>Score 0.50 - 0.75 0 Nodes (3%) 42 - 26 >>Score 0.75 - 1.00 1 Nodes (4%) 36 - 20 >>Score 1.00 - 2.00 0 Nodes (5%) 31 - 19 >>Score greater 2.00 1 Nodes (9%) 12 - 13 >> >>Compare on <A>ll <M>ove <S>core <T>ime <N>odes <Q>uit >> >>================= >> >>In this specific case (it is just a random example) I have tested a change in >>move-ordering swapping the order of killer-moves to see if it gains anything. >> >>The report says that 260 positions have been searched, that the change produces >>5 different moves, 8 different scores outside an interval of 0.10, 22 times >>outside an interval of of 10%. Of course intervals are flexible. >> >>Then the report shows that the change is good for a 1% speedup (ahem) in time as >>well as nodes. The "Nodes-Overview" (faster vs slower) reveals that there are >>more positions that run faster than slower which in this specific case is an >>important piece of information because the speed gain is so low and from another >>analysis function I could see the 1% was polluted by 2 positions that ran much >>longer and by skipping them the speed gain was higher. The consistency as shown >>in the nodes-overview becomes a dominant factor to accept a change (or not) in >>doubtful cases like this one. >> >>Another function (which I think every serious chess programmer should have) is >>the "Nodes-compare" function. It will report every difference in nodes of the >>tested positions, 260 in this case. It has been extremely useful for me to avoid >>bugs. >> >>Ed >Do you mean the whole tree? I don't do that, I just store to total-nodes-searched for the comparison later. >I can save the whole tree with node information but I don't use it to compare >different versions. >If you change the move ordering, for instance, the number of different nodes >will be huge, wont it? Yes of course, the node-compare function is not useful for regular changes but it becomes handy if you are rewriting stuff (faster code), conversion of C to assembler and then make the comparison to see if you haven't made errors. It's also useful when you have your engine fully parameter driven, connect every change you make to a parameter (or #define). Together with the node-compare function you can go back xxx versions and check if all is still okay (no differences). I can now go back to a version of 2 year ago and it still produces the same node counts while having made massive changes in the between time. It more or less guarantees that the new stuff you have programmed doesn't bite the old stuff. I can not emphesize enough the importance, as everybody knows a typo or bug is easily made. My best, Ed
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.