Author: Uri Blass
Date: 10:27:20 08/25/01
Go up one level in this thread
On August 25, 2001 at 13:19:57, Uri Blass wrote: >On August 25, 2001 at 05:20:42, Frank Phillips wrote: > >>On August 25, 2001 at 00:25:27, Christophe Theron wrote: >> >> >>>Yes. After Paris I have started to STOP relying on my personal feelings about >>>how my program should play. >>> >>>I have started to build objective tools (in which my subjectivity could not >>>influence) to evaluate my program and evaluate if changes were good or not. >>> >>>So since early 1998 I do not study closely lost games anymore, unless I can see >>>a clear repeated pattern in these losses. >>> >>>Since I have started to use this method, my program has dramatically improved. >>>So I do not see any reason to return to the old way of doing it. >>> >>>That's why I'm not going to give any particular importance to the losses that >>>happened during this WMCCC. That's my usual way of handling this... >>> >>> >>> >>> Christophe >> >>Christophe >> >>With all the information available, xboard, help from Bob and Bruce, fast >>computers and sustained effort, it seems reasonably straight forward to write a >>decent chess program that can beat most humans on the planet. Stepping up into >>the class of the commercials (and Bob and Bruce) is a different matter – >>something you obviously did with great success, without having to rely on >>superfast hardware. >> >>Are you prepared to give any concrete details about your approach to testing? >>It has clearly been successful - certainly much more so that my random changes >>inspired by disappointment at yet another loss. >> >>Frank > >I do not know christophe's way to test but I believe in test positions when it >should take a lot of time to build the test suite(I do not suggest tactical test >suites) > >My idea is to take some hundreds of positions from games and give some top >programs to analyze them for some hours. > >After doing it you can write for every position the moves that the programs >found after some hours. > >These moves are the possible solutions of your test suite(you can reduce the >number of solutions if you are sure based on analysis with the programs that >part of the programs are wrong). > >The way that I suggest to test if a change is positive or negative is simple. >give your program some minutes to ponder on every position and count the number >of solutions that your program found. > >If your program find more solutions the change is probably a good change. >If your program find less solutions the change is probably a bad change. > >It may take a year to build the test suite. >I also do not know if this idea works but I think that it works. > >Uri I can add that I think that it may also be important to use some test to test that you have no new bugs and for this reason using some tactical test may be a good idea. If your program find more solutions but have tactical bugs that cause it to do horrible things in 2% of the positions then it is not good so I think that using some simple tactical test suite may be also a good idea. nothing from what I say is based on experience. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.