Author: Robert Hyatt
Date: 15:29:35 01/17/98
Go up one level in this thread
On January 17, 1998 at 12:25:05, Stuart Cracraft wrote: >On January 16, 1998 at 17:42:01, Robert Hyatt wrote: > >>I can post 'em... but I really think WAC has become an antique... it >>is too easy. The only problem I don't see any way of solving without >>the >>full singular extension algorithm I used in Cray Blitz is wac230. >>Crafty >>simply won't see this one. The other 299 are not that difficult. If we >>set a 10 second limit and toss out the ones that can be found there, we >>end up with maybe 15 or so... >> >>I'm working on the ECM suite, which is way too big. But I'm going to >>end >>up with a hard but not impossible suite of around 300 positions that >>will >>be a good suite for a while... > >I agree and disagree. For people who have been doing this for years >with Win-at-Chess or already had strong programs when they started >using it for testing, then I agree. WAC would be worn out. > >But for others, Win-at-Chess is challenging. For example, I took >all this activity up about a month ago, testing against WAC. >After about one month of very hard effort, my program is within >20 problems or so of Crafty's result on the same hardware. Up from >being about 50 problems below Crafty's result, on Win-at-Chess. > >This has been done without tuning individual WAC problems (which >I will have to ultimately do), but rather in doing general improvements, >retesting against WAC as well as Kaufman/Louguet and accepting only >improvements that improve the assessed rating without dropping the >score on WAC. For real game play, I don't think is is necessary a >good way to proceed. I'd rather run against an endgame suite, a middle- >game suite, a checkmate suite and at least two ratings test. So ideally, >I'd like one test that combines all of these into one and isn't just >solvable on fast Pentium's with great well-debugged software. I do all >my development on a 486 25mhz on a program that is still undergoing lots >of development and have no plans of getting faster hardware. > >But with this said, I'd like to encourage RH to put together a suite >that doesn't favor one part of the game more than it should and that >is comprehensive enough, but not too hard that it only challenges a >well-debugged program like his. Most people out there have programs >that probably score 10%-50% lower on WAC than RH's current program >on the same hardware. Putting in a much harder test could be very >discouraging unless it has some other value-added such as a) better >balance of all phases of the game not just emphasizing combinations >and checkmates and b) produces a ratings estimate. > >--Stuart remember that a test suite can test multiple features of your program. WAC tests its tactical acuity. I've always considered tactics the "easy" part of chess. What we don't have is a good suite that tests knowledge. I'd love to have positions that have a right move that is right because of positional considerations. But it must be created carefully so that a super-deep tactical search won't find out that the weak backward pawn can actually be won... Some of the Bratko-Kopec positions were like this, but there were a few that I disagreed with, Kmoch be damned. :) But they were nice in that deeper searches wouldn't help if you didn't understand the particular pawn structure nuance they were targeting...
This page took 0.01 seconds to execute
Last modified: Thu, 07 Jul 11 08:48:38 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.