Author: Brian Richardson
Date: 11:53:05 08/29/04
Go up one level in this thread
On August 29, 2004 at 12:34:13, Stuart Cracraft wrote: >On August 29, 2004 at 05:40:26, Uri Blass wrote: > >>On August 29, 2004 at 01:16:31, Stuart Cracraft wrote: >> >>>Can you characterize what you hypothesize are the things that >>>the top programs have that Crafty might not yet have that explain >>>any gap in performance, however slight? What is the gap between >>>Crafty and Junior or Shredder or any of the other perennial >>>winners of the various events? >>> >>>Stuart >> >>I think that one of the main disadvantages of Crafty is that it does not use >>enough forward pruning so it is outsearched by better programs. >> >>I think that not using checks in the first ply of the qsearch is another problem >>of Crafty. >> >>Uri > >My program does this but not optimally but I've compared in this way. > >When search() hands off to quiesce at depth <= 0 (the < is because >null move may produce a depth that is negative as I don't put a >hard depth > X rule on the prerequisites for null move), quiesce >then first things sees if it is in check. If so, it returns >the value of the main search at ply 1. > >When I remove this one feature alone from my program, my test suite >result drops more than with any other feature removed, excepting possibly >hash table. > >It is at least the #1 or #2 feature of the program for test suite result >although I would imagine it slows down the program in a normal game markedly >and have no idea how much strength is impacted. > >Perhaps others can comment. > >I don't plan to remove it. > >Stuart I had long been wondering how Crafty NPS could be so fast, at least relative to Tinker. So I started working on a very simple PCSQ table and material only evaluation and comparing NPS with the addition of various evaluation terms. Crafty with its full eval was still faster. So then I started to look at the search. I thought because Crafty does not look at q-search checks (and has special check evasion only move generation) that this might be a factor. Of course, there are significant other differences, such as Tinker does EGTB probes in q-search and Crafty does not, IIRC. So, I did a version of Tinker with only in check testing for the first q-search ply, and a special make move for subsequent q-search plys and not bothering with in check testing, and various combinations of things. It turned out that this increased Tinker's NPS by about 2-3x, and the simple PCSQ and material eval was finally faster than Crafty (although not by much, and still slower than Gerbil). Unfortunately, Tinker's test set and actual game performance was much worse. Then I started adding each old "classic" Tinker evaluation term to the "new" basic Tinker (with the in-check q-search) and doing some testing. I have been doing a mix of basic test set testing, then some self play and some gauntlet matches against various other engines (with the 20 Nunn2 positions to try to remove the opening book as a factor), and of course ICS play. The simple PCSQ only Tinker did better at very fast time controls, and much worse at anything at about level 0 5 3 or slower. Also, ICS ratings can swing +/- about 100 points just based on the mix of opponents. The rating will go up quite a bit when playing some stronger humans, and then a strong program will come along and "steal" ratings points, but I perfer not to have a restrictive formula, since the occasional loss against a much lower ranked opponent can help find things too, and just playing some games is better than no games, IMO. I started with the pawn evaluation terms, and spent a lot of time on passed pawns, including distant opposition code. Many of old, long-time evaluation terms that I tested individually did not seem to help, and just slowed things down. I am also trying removing futility pruning and adding aspiration windows and some other things. So, I am still careful to keep an eye on NPS, but will trade pure NPS speed for a good additional evaluation term. Incidentally, it turns out the PCSQ terms are expensive in time relative to just about any other terms (except king safety attack square control, or mobility), but not worth doing incrementally, again for Tinker. So I only do pawn PCSQ tems (which don't cost much with separate pawn hashing), and some king endgame PCSQ, but not for other pieces. Brian
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.