Author: Andrew Wagner
Date: 15:25:20 03/31/04
Go up one level in this thread
On March 31, 2004 at 14:00:27, Russell Reagan wrote: >On March 31, 2004 at 13:23:23, Andrew Wagner wrote: > >> 1.) No move generator can get through this test without being right. > >At first thought, this seems like a bad idea to me (assuming it is correct after >only 100 test positions). You are testing 100 cases out of some huge number and >then assuming that part of your program is correct. If there is a bug in your >move generator later on, you will look at everything except your move generator >because you assumed that was working correctly. > >I think that if you used some test suite like this along with many other tests, >then you might be able to have confidence in your move generator. This test >suite idea sounds like a good way to find major bugs in a move generator. But >passing this test suite doesn't really prove a move generator's correctness, and >proving correctness is what you are claiming by "no move generator can get >through this test without being right." > >I think you need to do several different tests to be confident that your program >is right. You could do a test suite like this, run simple test suites of mate in >ones and see if your program solves them all, run lots of perfts and compare >with other engines (automate something to run overnight or for a few days), and >read over your source code slowly, asking yourself questions like, "what does >this line of code REALLY say?" Lots of times I write something, then I narrow >down a bug to that line of code, and I realize that what I wrote doesn't do what >I intended it to do. It takes a lot of different approaches to be confident >about your program's correctness, and even then, it's hard to be 100% sure. I disagree. The beauty of its test like this IS its simplicity. For example, when I first started trying to fix my move generator, I was getting wrong perft numbers at a depth of like 4 or 5 from the main starting position. Somehow, out of about of two million positions, I was getting 2 illegal ones. Now how on earth do you solve something like that? On the other hand, If I had had this suite to run, I would have quickly seen that it only failed on positions where there were pawn checks on the board (apparently, there are only two positions from depth 5 from the starting position where the king manages to walk into a pawn check). I can see what you're saying about being reluctant to declare a portion of code bug-free. But this test literally tests every possible KIND of move. The whole point was to be simple, but broad. I challenge anyone to come up with a move generator that's wrong and yet passes this move generator test. I really don't think it can be done. And if you only want to include this test as one of a bunch of move generator tests....great, but it should still be simple to implement as a test. I have something like this for my eval too, so that after a major change, I can quickly run a test to make sure I haven't broken something major. Andrew
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.