Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Automatic Eval Tuning

Author: Miguel A. Ballicora

Date: 20:05:10 07/02/01

Go up one level in this thread


On July 02, 2001 at 12:59:21, Vincent Diepeveen wrote:

>On June 29, 2001 at 12:23:16, Steve Maughan wrote:
>
>Hello othello is a children game compared to chess of course.
>It's like when i'm in the draughtsworld and people there still
>haven't figured out to cache EGTB probes in a special cache
>and they still haven't figured out there to do more probes
>in hashtable.
>
>Same is true for Othello.
>
>Add to that that Othello is nearly solved.
>
>If you would be searching a bit faster & deeper, then
>you don't even need tuning for othello.
>
>it's like saying that a bicycle steering wheel is exactly
>what a formula 1 car needs to perform better as he does now,
>because it works fine for you on your own bicycle.
>
>How many strong programs are automatic tuned?
>
>ZERO

I does not mean that there won't be one in the future. In fact, there is
no physical reason that says that a manual tuning will be always better
than an automatic or semi-automatic tuning.

Regards,
Miguel



>
>I remember knightcap. Its automatic tuning was so pathetic
>that in the end it was giving away pieces to give its opponent
>a few checks. Crafty in those days had a pretty weak king safety,
>and search depths dominated in those days. So aggressive play from
>a 7 ply engine against another 7 ply engine worked quite well back
>then.
>
>If it would play crafty now you would see a 0% score and knightcap
>would seem a lot less well.
>
>In short it's always easy to conclude that something works if
>all you need is to get closer to a 50% score.
>
>If you want to get a score > 50% against someone (for example
>an equal rated program) then you will run into trouble!
>
>>Vincent,
>>
>>>The bigger your evaluation is, the more problematic tuning it automatic
>>>is.
>>
>>True - as the domain space increases it is more difficult to find the optimal.
>>
>>>Also automatic tuners don't have any chess knowledge, so they
>>>don't see the difference between tuning passed pawns negative if you happen
>>>to have a testset where a passer is bad now and then.
>>
>>You simply need a large test set.  This is the same problem as solving a set of
>>equations.  In that case you have 'x' unknowns you need 'x' equations.  In the
>>case of an evaluation function you will need much more due to interaction.
>>
>>>Another problem for automatic tuners is that you tune for testposition set X,
>>>but that in reality it has to work well also for testset Y where it has
>>>not been tuned for.
>>
>>Again you need a large test set.
>>
>>>Evaluations hand tuned take into account testset Y, not only testset X.
>>
>>Not necessarily.
>>
>>>Anyway, when your number of parameters gets quite a big number then
>>>automatic tuning doesn't work anyway anymore.
>>
>>It is certainly more difficult.
>>
>>>Of course it might beat random chosen parameters, but it'll never beat
>>>hand chosen parameters (unless a fool choses them).
>>
>>This is not the case for some other games.  Othello being an example.  ALL
>>decent Othello programs (e.g. mine www.maughan.clara.net) are automatically
>>tuned.  Othello is a purely positional game and lends itself to self tuning.
>>Chess is MUCH more complex but I predict that in the next ten years we will see
>>some significant advances in automatic evaluation tuning.
>>
>>Regards,
>>
>>Steve Maughan



This page took 0.02 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.