Author: Odd Gunnar Malin
Date: 01:29:08 12/16/03
Go up one level in this thread
On December 16, 2003 at 01:41:15, martin fierz wrote: >On December 15, 2003 at 16:09:44, Dann Corbit wrote: > >>On December 15, 2003 at 16:01:17, martin fierz wrote: >> >>>On December 15, 2003 at 13:58:02, Dann Corbit wrote: >>> >>>>Number of tests: 183 >>>>Max score: 183 >>>>Engines score: 165 >>>>Number with full score: 165 >>>>Found in: 1 sec = 28, 2 sec = 11, 5 sec = 29, 10 sec = 20, 30 sec = 30, 1 min = >>>>9, 2 min = 12, 5 min = 17, 10 min = 4, 30 min = 5, >>>>Time used for tests (wrong answer=900s): 27361s >>>> >>>>Positions with error (id): >>>>ECM.1068, ECM.1197, ECM.1204, ECM.1217, ECM.1227, >>>>ECM.1229, ECM.1265, ECM.1273, ECM.1294, ECM.1308, ECM.1314, >>>>ECM.1438, ECM.1533, ECM.1592, ECM.1612, ECM.1617, ECM.1621, >>>>ECM.1622 >>> >>>BTW, how do you get these results? i use crafty 19.06 within arena on an athlon >>>XP1600+, and when i run ECMGCP at 5 seconds per move arena tells me that crafty >>>solves 91 of 183 positions. is there any difference to 19.07? what are you using >>>to run the EPD? >> >>Your machine is much faster than the one that I used. > >i know, i know. but yet, you are seeing MANY more positions solved, something >cannot be right with either of our setups. the draw detect option will have a >negligible influence in this test suite and cannot explain the huge discrepancy. > >>I am using gradualtest. >so what does that do? does it count the position as solved if the correct move >ever shows up, and gets lost again? arena will check after the allotted time, >and even if you were showing the move once, it will take the one you're >currently showing; which makes sense. > >cheers > martin There are two possible difference between GradualTest and other such tools but this should not give any different result if there isn't any error in the engine. 1. The cause of the utility, you can have several moves as bestmoves with different score for each move. 2. It use 'go' to start the search and '?' (move now) to terminate the search. This is for any bug in the program when used in a game also should be reflected in the test. It also would automatic go to the next position if the engine do a move when it find a mate etc. and is designed to do this. Eg. you don't need to wait for the timeout if the engine in real play neither would wait. For only counting the last change of prefered move should be the same as in other tools. NB! Crafty in analyze mode (other tools) and in playing mode (GradualTest) could give different results. Odd Gunnar
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.