Author: Miguel A. Ballicora
Date: 14:58:14 08/02/01
Go up one level in this thread
On August 02, 2001 at 16:23:54, Dann Corbit wrote:
>On August 02, 2001 at 16:06:00, Uri Blass wrote:
>
>>On August 02, 2001 at 14:47:20, Dann Corbit wrote:
>>
>>>On August 02, 2001 at 13:00:27, Miguel A. Ballicora wrote:
>>>
>>>>I have an idea for debugging that probably was used before but I want some
>>>>opinions about it:
>>>>
>>>>I am thinking about a system to trace what is going in my engine when I see a
>>>>weird move during a game. The main problem is that it is extremely difficult
>>>>to reproduce the same conditions for search. I have seen once in a while a
>>>>very weird move that I cannot reproduce it if I set the position to search.
>>>>The obvious reason is because the tables, hash and history bring information
>>>>that I lack when I try to reproduce.
>>>>Clearing the tables before each move would be (I think) an easy way out. The
>>>>engine in this state will be easy to debug but I would be dodging the problem.
>>>>In that way I would not see problems caused by the tables. I would be debugging
>>>>probably only part of the engine, dodging the most prone to errors!.
>>>>Saving the tables is out of the question, I would fill my HD in no time.
>>>>
>>>>What I think I can do is to save how many nodes I searched in any particular
>>>>move. When I ran out of time I keep record of the nodes.
>>>>When I try to reproduce the game I set the limit of thinking in "nodes", rather
>>>>than "time". I should be able to reach exactly the same situation in the search
>>>>in each of the moves of the games. Moreover, I should be able to reproduce the
>>>>whole thing with the debug version. Maybe I can store a log with all this
>>>>information and after the game let the program chew it and see if the
>>>>reproduction is exact.
>>>>
>>>>I think that debugging is tedious, boring but extremely important to make the
>>>>engine play the way we want. Particularly for chess programs where
>>>>the bugs are nasty because sometimes the program does not crash, it just plays
>>>>worse "sometimes".
>>>>
>>>>I just came up with this idea this morning, is there anything that I am missing?
>>>>suggestions? Is it obvious and everybody already does this?
>>>>
>>>>Regards,
>>>>Miguel
>>>>PS: I think that this kind of testing could be done also with pondering!
>>>>just limiting the pondering to n = nodes.
>>>
>>>I think to reproduce the exact state of a computer program will be nearly
>>>impossible if there is any way for randomness to be introduced (and surely there
>>>will be such).
>>
>>Why do you think that there will be randomness?
>
>There is always some randomness, even when it is not intentional.
>A program runs once and no other programs are in memory.
>On the next run, an IRQ fires and registers have to be pushed and popped. This
>shaves some bits off of floating point data.
I think I have no floating point variables that affect the search, I will check.
>On another run -- by bizarre chance -- srand(time(NULL)) happens to produce
>zero, and the rand() function performs incredibly stupid.
I have one srand(time(NULL)) that initializes the sequence of random numbers
used for the recently incorporated book. I think that I can do somthing like
this:
if (debugging_mode) {
srand( retrieve_from_log_previous_game(seed));
} else {
time_t seed = time(NULL);
log_for_debugging(seed);
srand(seed);
}
and I should get the same sequence of numbers, hence, the same book moves.
>On yet another run a program takes a huge slice of CPU. Because of this, the
>program takes a different branch than normally and plays a different move.
That is why I have the idea to stop the search based on "nodes searched". It
should be independent of the time. If the search stops exaclty at the same
point, I should get the same state of the tables and so on for the whole game.
Well, at least this is the idea.
I should store an array with this info
move move nodes
e4 e5 -book
Nh3 d6 123423
Be2 Be6 342321
d3 Qg5?? 234535
So if I try to debug Qg5 the idea is to make the program play e5 from the book
using the seed that I logged in the previous game. Then, search d6 and stop at
123423 nodes, later search Be6 with 342321 nodes as a limit, etc.
>Everything is random (to some extent) even what I am going to type next.
>
>>>
>>>It's a good idea to try, but don't expect miracles.
>>>
>>>Normally, if I want to reproduce something, I just take the EPD position where
>>>the problem occurred and let the program analyze. If it *never* makes the same
>>>choice it made before at any point in time, you have some very serious problem
Well, I have that problem with one position I found the other day. I wonder if
it is because it carries information from the hashtables from the previous
moves. It could be as you say, something serious, but I cannot test it since I
cannot reproduce that search. When the computer does this to me
I get nervous and I take it personally :-)
It could be a silly bug somewhere that forces me to use an uninitilized
variable... who knows, but I have to do something. If I can build a system that
I can reproduce this, I will have to pray that happen again and reproduce it.
>>>anyway (or a large amount of deliberately introduced randomness perhaps).
>>
>>I disagree.
>>It is possible that in the game it learned information from previous search and
>>this is the reason for the fact that it does not produce the same choice again.
>
>In this case, you delete the state information files (learning info).
I would plan to start with the same information file.
Regards,
Miguel
PS: As a test, I think that I will do a search and keep a record of the
nodes. I will repeat it with the same node limit and in both cases I will save
all the hashtables in one file and then compared them byte by byte.
This page took 0.24 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.