Author: Uri Blass
Date: 05:15:54 12/13/02
Go up one level in this thread
On December 13, 2002 at 07:08:32, Uri Blass wrote: >On December 13, 2002 at 03:17:43, Uri Blass wrote: > >>On December 13, 2002 at 01:08:26, Christopher A. Morgan wrote: >> >>> >>>50 Test Positions, 15 Engines - Results, Comparisons >>> >>>Recently, Gian-Carlo Pascutto, I think it was, posted about 150 position >>>problems, together with solutions, each with an identifying number of >>>ECM.xxx(x). I took the first 51, discarded one, and ran time to solve tests on >>>15 different chess engines for each of the 50 positions in the Fritz 7 GUI Tools >>>–> Analysis –> Process Test Set window. Below are the results of the tests. >>>The problems in FEN, as previously posted, follow the results. >>> >>>Some details: For each of the problems I confirmed the solution by letting a >>>couple of engines run individually in infinite analysis mode for 5-10 minutes. >>>The one problem discarded had three different solutions by four different >>>engines. For four of the problems, numbers 14, 28, 29 and 38 I listed the text >>>move as the solution together with a variation. In one case Nimzo 8 was only >>>engine of four I tested with that came up with the text solution. The other >>>three engines agreed on a different solution which became the variation. For >>>the other three positions, multiple engines agreed on a different solution which >>>became the variation. >>> >>>My goal was to have the majority of the engines solve every problem, so it would >>>be a test of how quickly a particular engine solved a problem compared to all >>>other engines in average speed of finding solutions to all problems, rather than >>>running for ten minutes (maximum allowed time per position) and not finding a >>>solution. >>> >>>Hardware: Athlon 750, 384MB RAM, 144MB RAM hashtables, except for Chess Tiger >>>14, and Gambit Tiger 2 which, apparently, only allow a maximum of a 96MB >>>hashtable. The times given should only be looked at in relative terms, that is >>>relative to the other engines. Faster processors will get much faster times, >>>but I would expect that the relative percentage differences in average speed >>>should remain constant among different processors running the same problems with >>>the given solutions. >>> >>>Problems 1, 24, 31, 33, and 41 took the most time for most engines, and a few >>>were not able to find a solution for some of these in the maximum ten minutes >>>allowed. >> >>I am surprised because 1 is one of the easiest problems in the test suite and >>movei finds it in less than one second. >> >>1 can be also solved for the wrong reasons(movei likes it at depth 2-4 only to >>change it's mind at depth 5 and to change it's mind again at depth 6 and I >>remember that one of the weakest engines showed Bg4 from the first ply) >> >>There is no doubt that movei can see clear tactics at depth 7 and the score is >>+3 after only 1.38 seconds on p850. >> >>The main problem with 1 is that d4 is another way to win but I expect all >>engines to see a better score for Bg4. >> >>24 is not very easy like 1 but it is also not hard to solve. >> >>103 seconds and depth 10 on p850 is enough to see fail high on Nxh7. >> >>The hard problem is 28 and I believe that programs usually solve it for >>positional reasons. > >I now see that you changed 28 to have 2 solutions so by your definition it >solved it. > >I checked latest movei and it solved all the positions in 250 seconds per move >except one of the hard positions. correction there are 2 exceptions. I noticed that the positions that are used are different than the position that I use and position 41 was not included so except positions 31 and 41 latest movei solved everything in less than 250 seconds. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.