Author: Ed Schröder
Date: 05:00:56 09/25/04
Go up one level in this thread
On September 25, 2004 at 07:29:54, martin fierz wrote: >On September 25, 2004 at 06:36:04, Ed Schröder wrote: > >>On September 25, 2004 at 04:24:46, martin fierz wrote: >> >>>aloha! >>> >>>i made a small experiment: old root move ordering vs new root move ordering. >>> >>>old: >>>generate all moves. do not order. use normal search and after each completed ply >>>(or fail high) move the current best move to the top of the list and shift all >>>moves back. >>> >>>new: >>>generate all moves. do not order. use normal search and after each completed ply >>>(or fail high) move the current best move to the top of the list and order the >>>remaining moves by subtree size. >>> >>>results on centrino 1.4GHz: >>>- test set ECMGCP 5s/move: old 107/183 solved, new 103/183 solved >>> >>>- matches at blitz 1'+5'' increment vs frenzee & gothmog: >>>old: 8-32 against gothmog, 21-19 against frenzee >>>new: 7-33 against gothmog, 20-20 against frenzee >> >>This is a terrible way of testing. >i feel stupid :-) he he.... >in fact, i have a test mode in my program which does exactly what you suggest >(only my test file doesn't have enough positions yet i'm afraid). >the result on 33 positions is new = 100.9% of old nodes searched. > >i will increase the number of positions and report back. thanks for pointing out >how stupid my test is! 300 positions is a bit overdone, I think 200 is good enough. It's my experience with move-ordering that a new move-ordering idea is only an improvement when the idea produces less nodes/time most of time, a steady behaviour. It's not such a good idea to test 200 positions and decide on the total nodes (time) searched. For instance, when 130 positions (65%) of the positions produce fewer nodes and the total nodes (time) is higher then the change most of the time is an improvement after all. >in my defence i will say that as a working citizen, i don't lose time by >performing such a test - the match runs over night and while i work; for a >professional chess programmer it is probably more important to do tests fast, i >can make more tests than programming, for you it was probably the other way >round. doesn't change that you are right of course! Nowadays testing is a pain, in the early days with processors running at 5Mhz not exceeding 2000 elo at 40/2h testing was easy, an improvement could be measured with the naked eye soto say, but how do you measure an improvement when your program is playing at 2700? How do you measure the difference between 2700, 2690 and 2710 when your own level is only 1700-1800? Well, you need lots of PC's :) Ed
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.