Author: Vincent Diepeveen
Date: 04:44:15 07/05/01
Go up one level in this thread
On July 04, 2001 at 09:12:35, JW de Kort wrote:
>On June 29, 2001 at 14:05:06, Jon Dart wrote:
>
>>On June 29, 2001 at 11:14:34, Artem Pyatakov wrote:
>>
>>>I am curious, have people here experimented or extensively used Eval Function
>>>tuning based on GM games for example?
>>>
>>>If so, is it effective to any extent?
>>
>>There is a program called KnightCap that implemented eval
>>learning and it worked quite well. Source is available.
>>See http://syseng.anu.edu.au/lsg/knightcap.html.
>>
>>--Jon
>
>Hi i tried to understand what they are doing by reading a paper on their site.
>But being a tax lawyer my mathemetics bothered me. Can someone please explain
>their methode in language that is understandable to some one with only high
>school mathematics.
>
>Thanks
>
>Jan Willem
Oh well scientific world is going to kill me (the AI part),
if i say this, but all those learning things are closest to
next thing:
In a program there are values, for example open file might
get 0.20 pawn bonus.
All those bonuses and penalties we call parameters.
Automatic tuning, whatever form more or less is going to randomly
guess a different value for every parameter.
Then the program plays a few games, or most likely only does a few
tests. Based upon the result of the test then gets decided whether this
tuning is better or worse.
Obvious problems
- if the way of testing is very primitive, so will the conclusions
about a parameter set be
- less obvious, but hard reality is that from game playing viewpoint
the best versions of diep always were versions which did less well
on the testsets. Too aggressive evaluation tuning solves of course
way more positions but there is every move perhaps 50% chance that
you play a patzer move and lose the game because of that, whereas
a more positional alternative is more likely to winning the game
chanceless.
- No independant conclusions get drawn like humans can.
Like we watch a program play and say: "he it is getting way too much
points for open files, it goes to the open files quick but neglects
the rest of the play!"
- So if there is 1 parameter in an evaluation,
then the automatic learning needs
to try all values for that 1 paramater before it knows whether it's
Suppose we have 2001 values for 1 parameter (-1000 to +1000), then
that's 2001 experiments. Now for 2 parameters it would need
2001 x 2001 experiments.
Obviously for 100 parameters it needs 2001^100 then
for 1000 it needs 2001^10000. That's a number which will not fit
soon on your paper so many zero's it has.
etcetera.
In short automatic learnings weak point is the huge number of experiments
needed. Obviously every researcher in automatic tuning focuses upon
methods to get that number of experiments as
small as possible.
Till today however no one succeeded in getting it lineair. To do that
you need intelligence and domain dependant knowledge.
That last must not be underestimated, but the first still gets hugely
underestimated by non-scientists.
The learning program is usually using very simplistic algorithms to
get to its numbers.
The average chessprograms search is usually way smarter as the most
complicated learning which is producing results somehow.
The trick 100% of all researchers use to show their learning works
is to compare with the most insane parameter set so called 'handpicked'.
If i would only tune for a day, then let the automatic tuning of knightcap
learn and learn and learn for over a year or 10. Then play
knightcap-diepeveen tuned versus knightcap-selftuned,
then we would see a huge score difference. Of course if i were
the knightcap programmer i would start with a bonus for doubled pawns
and a penalty for putting rooks to open files. In that way i could
conclude that learning works.
This is how 100%, can be 99.9% too, of the conclusions are about
learning.
Way too optimistic. It's really a childish thing!
Now it's unfair to say something bigtime negative about knightcap,
because the level of this experiment is already way above other
tuning experiments. The average tuning experiment is way smaller setted
up, and if such a childish average experiment gets filmed by discovery
channel or whatever, then they also directly conclude that in future
robots are going to take over.
This page took 0.05 seconds to execute
Last modified: Thu, 07 Jul 11 08:48:38 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.