Author: Bob Durrett
Date: 08:50:29 11/16/00
Go up one level in this thread
On November 16, 2000 at 10:43:42, Uri Blass wrote: >On November 16, 2000 at 10:29:02, Robert Hyatt wrote: > >>On November 16, 2000 at 09:14:38, Peter Berger wrote: >> >>>On November 16, 2000 at 09:07:23, walter irvin wrote: >>> >>>>to me programs fall into 2 list bean counters and knowledge based . >>>>bean counters >>>>fritz >>>>junior >>>>nimzo >>>>lg2000a >>>> >>>>knowledge based >>>>shredder >>>>hiarcs >>>>rebel >>>>tiger >>>>diep >>>>crafty >>>>king >>>> >>><snip> >>> >>>Could you post an example for a position where for example Crafty shows superior >>>chessknowledge compaired to Junior ? >>> >>>pete >> >> >>Since I don't have any commercial program, I can't test them, but here is one >>simple test, just for fun. Evaluation after 1 ply search... >> >>[D]4k///3pp/1P4P///4K w >> >>This is just one test, of many. The idea is that last year, a GM friend of >>mine tried this against a couple of well-known programs, and both preferred >>black with the two connected passed pawns. This would be correct if there >>were lots of pieces left on the board. But in this position, the white king >>can stop the black pawns easily. The black king has a problem. >> >>It would be interesting to see how other programs handle this. My static >>eval is +1.5, for instance, although a 1 ply search will produce the same >>score, approximately. >> >>If you want, we can try a few of these for fun, for anyone willing to >>participate. I hate to give away all my 'secrets' of course, but since Crafty >>is open source, nothing is really secret. :) > >The evaluation of my Junior5.9 after 1 ply search is +0.58 for white >The evaluation of hiarcs7.32 after 1 ply is +5.27 for white >The evaluation of fritz5.32 after 1 ply is +0.03 for white > >I think that it is better to take the following position for testing > >[D]4k3/8/8/3pp3/8/8/1P4P1/4K3 w - - 0 1 > >Uri The use of test positions seems to be the only practical way to advance a chess-playing program from an immature state to maturity. Fritz, for example, would be "mature" whether or not it should be classified as "knowledge based" or "bean counter." However, the use of test positions to determine DIRECTLY whether or not the program is "knowledge based" or "bean counter" will be complicated by the fact that the tests using test positions may reveal more about the programming skills and chess knowledge of the programmer, and about the maturity of the software, than it would about how the software should be classified [i.e. "knowledge based" vs "bean counter."] Another measure which might be considered is the number of lines of code [in the source code] dedicated to position evaluation, assuming one can find out this information. For example, a program with several lines of code devoted to position evaluation might be [perhaps arbitrarily] declared to be "knowledge based" whereas one with a few hundred or less might be called "bean counter." The problem here is that the source code may not be available. You would have to ask the programmer. He/she might be willing to answer that question even if unwilling to reveal the code itself. In this way of measuring, the maturity of the software and the quality of the code would not influence the answer [i.e. "knowledge based" vs "bean counter."] A new untested program, produced by a programmer relatively new to the programming of chess-playing software might still be "knowledge based" even though the program might still play poorly due to it's immaturity. The bottom line seems to be that there is no perfect way to determine DIRECTLY whether or not the program is "knowledge based," "bean counter," or something in between [a hybrid?]. Instead, it seems necessary to use inference, based on some indirect measure(s). This suggests that several different measures be used and the conclusion drawn from the composite results. Another indirect measure, from which inferences might be drawn, is the style of play. Here, test positions seem the way to go. One might use "tactical" chess positions and "positional" test positions. One might infer that programs that do well at "tactical" positions but do poorly at "positional" [or "closed"] positions would be "bean counter" and if the opposite were the case, then the program might be inferred to be "knowledge based." One must still wonder how important the issue really is.
This page took 0.02 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.