Author: Andrew Wagner
Date: 16:24:37 03/31/04
Go up one level in this thread
On March 31, 2004 at 19:06:00, Russell Reagan wrote: > > >If you get a wrong perft 5, you run a perft 4 for each of the root moves and see >which ones are wrong. Only one or two of those will be wrong. Then you continue >narrowing it down until you find the offending position. You can automate it if >you wanted to modify Crafty to do the same thing. You could let that run all >night and you could have a file full of positions that your engine handles >wrong. You could fix something, and then run that file through your program like >an EPD test suite and eliminate bugs one by one until you calculate perft for >all of those positions correctly. > > >>On the other hand, If I had had this >>suite to run, I would have quickly seen that it only failed on positions where >>there were pawn checks on the board (apparently, there are only two positions >>from depth 5 from the starting position where the king manages to walk into a >>pawn check). I can see what you're saying about being reluctant to declare a >>portion of code bug-free. But this test literally tests every possible KIND of >>move. The whole point was to be simple, but broad. I challenge anyone to come up >>with a move generator that's wrong and yet passes this move generator test. I >>really don't think it can be done. And if you only want to include this test as >>one of a bunch of move generator tests....great, but it should still be simple >>to implement as a test. I have something like this for my eval too, so that >>after a major change, I can quickly run a test to make sure I haven't broken >>something major. Andrew > > >You may be right. However, I think your test fails to find the obscure bugs. If >you are just not generating one type of move correctly (ex. castling moves), >then your test will probably find that. Sometimes you have crazy bugs though, >like you don't generate one specific move (as opposed to one type of move), like >maybe you don't generate pawn capture moves which are also promotions to a >knight on h8 because you made a typo and your program considers h8 to be off the >board (but only in the case of a pawn promotion-capture to knight on h8)? Your >test may handle that one, but may not handle some other off the wall case. You >can't be sure unless you proved it. > >I am not saying your test is bad or that it won't provide useful information. It >is just not a good idea to say something can "never" happen unless you have >proof that it can never happen. It may be possible to do a proof by cases to >show the correctness of a test like yours. However, I think it may require way >more than 100 test positions. Ok, this makes more sense. I understand where you're coming from when you say it's not good to ever assume a section of code is perfect. Probably the strength of a suite like this is for the first level of debugging (when you first write the move generator and forget to include promotion captures or something), and also as a test when you're not necessarily looking for a bug, and just want to make sure you didn't break anything major.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.