Author: Stuart Cracraft
Date: 14:28:16 01/19/98
Go up one level in this thread
On January 17, 1998 at 17:28:53, Bruce Moreland wrote: > >On January 17, 1998 at 15:45:59, Stuart Cracraft wrote: > >>The reason is that I've not had enough time to debug these and >>make them efficient or even determine their efficacy. When I took out >>extension set "A" above, and reran on Win-at-Chess with no extensions at >>all, the score on Win-at-Chess dropped to 123 solved out of 300, a drop >>of 69 problems solved (from 192 out of 300 with extensions). This >>is a drop in total WAC correct from 64% to 41% using my >>limited time (5 seconds) and limited hardware (486) experiment. > >You can get more data out of these runs. > >You can record at what point a problem was solved, so you know exactly >when everything was solved. > >Then, rather than just grepping your log file for "right" or "correct" >or "success" or whatever it is, you can show how many were solve after >1, 2, 3, 4, and 5 seconds, rather than just 5 seconds. You can graph >this and get an even better idea of how well extensions work. With 5 >seconds this isn't very useful, but if you do runs of a minute or two >you sometimes see interesting curves. Often you get two versions that >solve the same number, but one of them will solve a lot more in the >first few seconds. And sometimes one will jump out to a huge lead, but >the curves will cross in the last ten seconds of a run. > >bruce Yeah -- I know. I gotta write an analyzer program to sift through this massive output and give me things like "the sum of the squares of all the times at which point the program first saw the solution and held to that solution through to the terminal iteration", as well as "the average of the first solution time." Another useful one would be "the average iteration at which first solution is found and subsequently stuck to." Yep, plenty of grist for the mill in such a program. A real programming aid for examining test runs... --Stuart
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.