Author: Peter Fendrich
Date: 13:15:53 05/24/04
Go up one level in this thread
On May 24, 2004 at 14:39:36, Ed Schröder wrote: >On May 24, 2004 at 13:09:42, José Carlos wrote: > >>On May 24, 2004 at 12:39:35, Ed Schröder wrote: >> >>>On May 24, 2004 at 12:07:56, Russell Reagan wrote: >>> >>>>On May 24, 2004 at 10:13:33, Ed Schröder wrote: >>>> >>>>>>It would also be interesting to hear some background - how you arrived at what, >>>>>>what else you tried & discarded, how confident you are about it, any general >>>>>>thoughts, etc. >>>>> >>>>>That would take too much to read, I prefer to stay on topic and as short as >>>>>possible. Your question is much better suited for a forum, so: I feel my way of >>>>>doing king safety is okay, I can't say that about the evaluation of passed >>>>>pawns, I feel most insecure about it. >>> >>>>It would be interesting to know how you determine if a change you make is an >>>>improvement (if you are allowed to tell us, of course). >>> >>>You guessed well, I can't tell. >>> >>>Ed > >> But Ed, you surely had a method to determine that way before meeting >>Christophe (which is the reason, I guess, you can't tell now). > >Yes. > > >>Could you comment >>on your pre-Christophe testing methods? >> Thanks in advance. > >I can tell you one part, here is. I have testsets that contain well selected >positions (tactics, positional, midgame, endgame etc.) Depending on the changes >I have made I run a specific testset (or testsets) as first step in my own >text-based DOS interface, it serves as a first smell. When finished I can >produce several reports, such as to compare the results with a previous version. > >A screenshot: > >Rebel ... version RXP 05-07-2003 >(REBEL XP) (A-2000) CV=100 > >Rebel ... version M02 04-04-2004 >(KILLER - KD1|KD2|KD1-2|KD2-2) CV=RXP > >Number of searched positions : 260 >Number of different moves : 5 >Number of different scores : 8 (Interval score = 0.10) >Number of different times : 22 (Interval time = 10%) > >Total time (version 1) : 00:57:56 Nodes 3.187.570 >Total time (version 2) : 00:57:32 Nodes 3.151.571 >Total percent : 1% (1.14%) > >Score 0.01 - 0.10 22 Nodes faster vs slower 119 - 117 >Score 0.10 - 0.25 5 Nodes (1%) 119 - 51 >Score 0.25 - 0.50 1 Nodes (2%) 61 - 34 >Score 0.50 - 0.75 0 Nodes (3%) 42 - 26 >Score 0.75 - 1.00 1 Nodes (4%) 36 - 20 >Score 1.00 - 2.00 0 Nodes (5%) 31 - 19 >Score greater 2.00 1 Nodes (9%) 12 - 13 > >Compare on <A>ll <M>ove <S>core <T>ime <N>odes <Q>uit > >================= > >In this specific case (it is just a random example) I have tested a change in >move-ordering swapping the order of killer-moves to see if it gains anything. > >The report says that 260 positions have been searched, that the change produces >5 different moves, 8 different scores outside an interval of 0.10, 22 times >outside an interval of of 10%. Of course intervals are flexible. > >Then the report shows that the change is good for a 1% speedup (ahem) in time as >well as nodes. The "Nodes-Overview" (faster vs slower) reveals that there are >more positions that run faster than slower which in this specific case is an >important piece of information because the speed gain is so low and from another >analysis function I could see the 1% was polluted by 2 positions that ran much >longer and by skipping them the speed gain was higher. The consistency as shown >in the nodes-overview becomes a dominant factor to accept a change (or not) in >doubtful cases like this one. > >Another function (which I think every serious chess programmer should have) is >the "Nodes-compare" function. It will report every difference in nodes of the >tested positions, 260 in this case. It has been extremely useful for me to avoid >bugs. > >Ed Do you mean the whole tree? I can save the whole tree with node information but I don't use it to compare different versions. If you change the move ordering, for instance, the number of different nodes will be huge, wont it? Anyway, for many changes it shouldn't change too much. Thanks for the tip! /Peter
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.