Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: What's best low BF or good WAC result?

Author: Richard Pijl

Date: 01:01:24 03/19/03

Go up one level in this thread


On March 18, 2003 at 14:37:19, Albert Bertilsson wrote:

>Hi!
>
>I just want to publish my findings with WAC and Sharper, as I've written
>previously Sharper managed a bad 195 out of 300 in WAC.
>
>Running test with the 105 failed positions gave this:
>
>Adding check extensions 69 of 105 solved, great! But adding check extensions
>really lowers the BF because a lot of extending is done on a single meaningless
>check. In Sharpers built in test node count rise with almost 40% =(.

The idea of extensions is to get deeper in interesting positions where something
can happen, and not examining the quiet positions where nothing happens. Of
course it is not easy to determine which checks are meaningful and which are
not, but I think that the gain of extending on checks is much higher than the
cost.

In the Baron extensions can make the search to go three times as deep in
(hopefully) interesting positions, spotting tactics sometimes much earlier than
it did before.

You can't compare a 6 ply search without extensions with a 6 ply search with
extensions and say my node count for the 6 ply search rises so I must do
something about it, as you search some positions deeper.
A nice exercise might be to check how much extra time, average ply depth and
nodes you need in sharper to get to 264 solved positions and compare that with
what you get with check extensions.

If you are adding more types of extensions you will need to look into ways of
limiting it though. With just check extensions I doubt that is necessary.

>
>So I add code to only extend on two checks or more, and I get this:
>52 out of 105 problems solved. Node count in the test rise with about 2%.

Are you testing fixed depth? I suggest you test using a fixed amount of time per
position. For WAC 5 sec's per position should be fine to start with.

>Clearly check extension are great to have.

I would say that it is mandatory for even an average strength engine.

>But they are not so great to have most of the game when they don't manage to
>give any better result. So I'm thinking maybe turn on check extensions in the
>end game, and have the two check rule during the rest of the game. However, this
>won't help much as many of the positions in WAC are not end game positions (at
>least not by Sharpers measure).

Don't let the 40% extra nodes scare you. You'll gain 17 solved positions with it
(compared to the two-check mechanism) ...

>It all comes down to, what's best low BF or high WAC result?

BF is rather meaningless if you're working on extensions. You should only worry
about BF when working on things like move ordering or pruning as this is about
reducing the work while getting the same result. With extensions you try to get
a better result with minimal extra effort (so BF might increase), but it is
extra effort by definition as you are 'extending' the search. Of course you
still need to test whether the cost for the better result is acceptable ...

Tactical testsuites like WAC can really deceive you though. I noticed quite soon
that better results on WAC does not mean that you have a better version of your
engine. Increasing kingsafety parameters will increase WAC performance in most
engines, but the number of incorrect sacs will increase during games. So before
making a decision about a change you've made you should also test in games.

>What do you think
>about extension rules based on game phase?

I wouldn't do this for check extensions, but for e.g. passed pawn extensions
that might be a good idea.

>I know that 53 or 36 failed positions is not anyway near the results you
>consider good, but remember that Sharper is weak engine with lot of work still
>to do. (And I'll do it, just give me some time)

With the Baron I had just over 200 solved positions (5 sec per position) 18
months ago and now it is solving almost 280 in 1 sec. Improvement is surely
possible!

Richard.



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.