Author: Russell Reagan
Date: 16:06:00 03/31/04
Go up one level in this thread
On March 31, 2004 at 18:25:20, Andrew Wagner wrote: >I disagree. The beauty of its test like this IS its simplicity. For example, >when I first started trying to fix my move generator, I was getting wrong perft >numbers at a depth of like 4 or 5 from the main starting position. Somehow, out >of about of two million positions, I was getting 2 illegal ones. Now how on >earth do you solve something like that? If you get a wrong perft 5, you run a perft 4 for each of the root moves and see which ones are wrong. Only one or two of those will be wrong. Then you continue narrowing it down until you find the offending position. You can automate it if you wanted to modify Crafty to do the same thing. You could let that run all night and you could have a file full of positions that your engine handles wrong. You could fix something, and then run that file through your program like an EPD test suite and eliminate bugs one by one until you calculate perft for all of those positions correctly. >On the other hand, If I had had this >suite to run, I would have quickly seen that it only failed on positions where >there were pawn checks on the board (apparently, there are only two positions >from depth 5 from the starting position where the king manages to walk into a >pawn check). I can see what you're saying about being reluctant to declare a >portion of code bug-free. But this test literally tests every possible KIND of >move. The whole point was to be simple, but broad. I challenge anyone to come up >with a move generator that's wrong and yet passes this move generator test. I >really don't think it can be done. And if you only want to include this test as >one of a bunch of move generator tests....great, but it should still be simple >to implement as a test. I have something like this for my eval too, so that >after a major change, I can quickly run a test to make sure I haven't broken >something major. Andrew You may be right. However, I think your test fails to find the obscure bugs. If you are just not generating one type of move correctly (ex. castling moves), then your test will probably find that. Sometimes you have crazy bugs though, like you don't generate one specific move (as opposed to one type of move), like maybe you don't generate pawn capture moves which are also promotions to a knight on h8 because you made a typo and your program considers h8 to be off the board (but only in the case of a pawn promotion-capture to knight on h8)? Your test may handle that one, but may not handle some other off the wall case. You can't be sure unless you proved it. I am not saying your test is bad or that it won't provide useful information. It is just not a good idea to say something can "never" happen unless you have proof that it can never happen. It may be possible to do a proof by cases to show the correctness of a test like yours. However, I think it may require way more than 100 test positions.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.