Author: Robert Hyatt
Date: 15:29:35 01/17/98
Go up one level in this thread
On January 17, 1998 at 12:25:05, Stuart Cracraft wrote: >On January 16, 1998 at 17:42:01, Robert Hyatt wrote: > >>I can post 'em... but I really think WAC has become an antique... it >>is too easy. The only problem I don't see any way of solving without >>the >>full singular extension algorithm I used in Cray Blitz is wac230. >>Crafty >>simply won't see this one. The other 299 are not that difficult. If we >>set a 10 second limit and toss out the ones that can be found there, we >>end up with maybe 15 or so... >> >>I'm working on the ECM suite, which is way too big. But I'm going to >>end >>up with a hard but not impossible suite of around 300 positions that >>will >>be a good suite for a while... > >I agree and disagree. For people who have been doing this for years >with Win-at-Chess or already had strong programs when they started >using it for testing, then I agree. WAC would be worn out. > >But for others, Win-at-Chess is challenging. For example, I took >all this activity up about a month ago, testing against WAC. >After about one month of very hard effort, my program is within >20 problems or so of Crafty's result on the same hardware. Up from >being about 50 problems below Crafty's result, on Win-at-Chess. > >This has been done without tuning individual WAC problems (which >I will have to ultimately do), but rather in doing general improvements, >retesting against WAC as well as Kaufman/Louguet and accepting only >improvements that improve the assessed rating without dropping the >score on WAC. For real game play, I don't think is is necessary a >good way to proceed. I'd rather run against an endgame suite, a middle- >game suite, a checkmate suite and at least two ratings test. So ideally, >I'd like one test that combines all of these into one and isn't just >solvable on fast Pentium's with great well-debugged software. I do all >my development on a 486 25mhz on a program that is still undergoing lots >of development and have no plans of getting faster hardware. > >But with this said, I'd like to encourage RH to put together a suite >that doesn't favor one part of the game more than it should and that >is comprehensive enough, but not too hard that it only challenges a >well-debugged program like his. Most people out there have programs >that probably score 10%-50% lower on WAC than RH's current program >on the same hardware. Putting in a much harder test could be very >discouraging unless it has some other value-added such as a) better >balance of all phases of the game not just emphasizing combinations >and checkmates and b) produces a ratings estimate. > >--Stuart remember that a test suite can test multiple features of your program. WAC tests its tactical acuity. I've always considered tactics the "easy" part of chess. What we don't have is a good suite that tests knowledge. I'd love to have positions that have a right move that is right because of positional considerations. But it must be created carefully so that a super-deep tactical search won't find out that the weak backward pawn can actually be won... Some of the Bratko-Kopec positions were like this, but there were a few that I disagreed with, Kmoch be damned. :) But they were nice in that deeper searches wouldn't help if you didn't understand the particular pawn structure nuance they were targeting...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.