Author: Dann Corbit
Date: 20:55:17 03/06/06
Go up one level in this thread
On March 06, 2006 at 23:39:36, Stuart Cracraft wrote: >On March 06, 2006 at 22:47:18, Dann Corbit wrote: > >>On March 06, 2006 at 22:13:56, Stuart Cracraft wrote: >>[snip] >>>Discouraging Dan. Discouraging. >> >>Suppose that you are 3 lines away from approximately the same result. >> >>BTW, I have had a crafty version score 300/300 with the same time controls. >> >>The only difficult problem in this set is WAC.230. >> >>I think that WAC is a great set to start working with on a chess engine. >>After a few months you are going to graduate to something tougher. >> >>I guess that there is some simple bug that is costing you 80% or more of the >>misses. >> >>It sounds like an advanced engine from the things I have read so far. >> >>I think I saw a list of the missed problems by your program. I guess that I may >>see a theme problem when I go over them. > >Thanks - I look forward to those comments at your availability. > >I reran the whole 300 suite at 10 seconds each this evening, >due to Bob's comment, and pulled the failed 29 (Bob failed 9 >I believe.) > >Then I reran against just my 29 and found that only 24 failed the >second time, at the same time control. > >This tells me that there is something about the test that is not >reproducible, based on either the ordering of the tests in the suite >or aspects being carried from test position to test position (hash >tables, history heuristic, etc.) I am not sure what it is. > >To test this theory, I took the 29 that failed of which only 24 failed >the second time and reversed them so that the last came first and retested. >This time 4 of the 29 were solved instead of 5. This difference of one is >too small to claim an ordering result for just 29 position sample size. > >Still this indicates that instead of failing 29 I am failing 24-25 >and I am not sure what would cause it. Before every iterative deepening >ply 1 search, I clear out the history heuristic table, the hash tables, >and the principal variation arrays. > >Still that is only 4-5 more leaving 24-25 left, arguably 15-16 if one >wants to aspire as high as Crafty. > >I am at a total roadblock on the subject. As I mentioned, I will be >putting money where my mouth is and making a signficant donation to >the board sponsors for guidance to a solution of gaining say another >10 right above my current 271 at 10 seconds. (Hopefully that's legal >here.) This sounds like a bug. If you analyze a new position, you should definitely clear the hash table between analysis runs. If the results are not exactly the same time after time, then there is a bug (unless you introduce randomness or use floating point for your eval or other odd things like that). Memory overwrite or hashing or something like that may be at fault. Do you have a way to compute the hash signature from scratch instead of incrementally? If so, I suggest doing a debug build that for every make-move computes the hash completely from scratch and then compares the hash signature with the incremental hash signature created by make-move. If anything does not match, you may have found the culprit. Also, valgrind or some other memory checking tool might prove valuable. If your code is ANSI C, you could run SPLINT against it. If you are not opposed to sending me a copy of your code I will take a look at it for you.
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.