Author: Uri Blass
Date: 10:07:36 01/12/06
Go up one level in this thread
On January 12, 2006 at 11:43:28, Marc Lacrosse wrote: >Not so long ago (one year maybe), when Shredder 9 was an undisputed leader of >the field, V. Pittlik published an interesting analysis showing that shredder >was an extremely fast evaluator : when analysing a series of difficult epd test >positions, shredder was not only one of those who solved the largest number of >quizzes when a long thinking time was allowed but it distinguished itself mostly >by a considerable superiority over all other programs when very short thinking >time was afforded per position. >This correlated very well with the number of brilliant victories won by shredder >in blitz tournaments. > >As I wondered how the newer stars would perform from this point-of-view, I >selected 87 difficult positions within various well-known epd test-suites >(ECMGCP, LPTII, ...). >I had them analysed by six programs at 10 seconds / position and then at 180 >seconds /position on a Pentium IV 2200Mhz PC. >The programs were Rybka 1.0 beta 32, Shredder 9 UCI, Fruit 2.2, Toga II 1.1a, >Ktulu 7.0a and Gandalf 6.0. > >For each program I evaluated the ratio between the number of positions solved at >10 seconds and the number of positions solved with 3 minutes thinking time per >position. > >This ratio is : > >Rybka 57 % >Shredder 57 % >Gandalf 55 % >Ktulu 53 % >Toga 39 % >Fruit 36 % > >So we see that Rybka is one of the fastest searchers (and its superiority should > specially pronounced at blitz) whereas Fruit and Toga are incredibly weaker >when a too short thinking time is given. > >Marc > >PS : the absolute number of positions solved by the diferent engines with 180 >seconds per move is : Rybka and shredder 68, Toga and Fruit 61, Gandalf 60, >Ktulu 55. As you can see rybka and shredder are the best in your test also at 180 seconds per move and the result may be misleading. The problem is that the ratio is misleading and it is possible that the task of the engines that solved more position to get the same improvement in number of solutions is harder. I think that you should test the minimal time that every program needs to solve 34 positions and the minimal time that every program needs to solve 68 positions and to divide the times for comparison. The program that the result of the division is closer to 1 is a good candidate to be relatively better at long time control. For example If shredder needs 7 seconds for 34 solutions and 140 seconds for 68 solutions when fruit needs 20 seconds for 34 solutions and 200 seconds for 68 solutions then it supports the theory that fruit is relatively better at long time control because fruit needs only to multiply it's time by 10 to achieve a target that shredder needs to multiply it's time by 20 and in this case fruit's ratio(200/20) is smaller than shredder's ratio(140/7) so it is closer to 1. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.