Author: Komputer Korner
Date: 05:49:32 07/31/98
Go up one level in this thread
On July 31, 1998 at 07:49:14, Amir Ban wrote: >On July 31, 1998 at 04:01:08, blass uri wrote: > >> >>On July 31, 1998 at 02:24:20, Jouni Uski wrote: >> >>>On July 31, 1998 at 01:51:57, blass uri wrote: >>> >>>>The programs I know give me evaluation in pawns and I prefer to see >>>>in the evaluation function the predicted result of the game(number between 0 >>>>and 1) and not an evaluation in pawns. >>>> >>>>Uri >>> >>>What are You meaning? Predicted result evaluation has only 3 values: 0, 0.5 and >>>1! Not very informative... >> >>I mean the average resultI expect to have >>for example it can be 0.7 if I believe the probability to win is 50% >>and the probability of a draw is 40% >> >>Uri >>> >>>Jouni > >You can see from a few responses you got, that the interpretation of the eval in >terms of expected result is not obvious to many. > >What are you asking for is not so simple. Suppose I printed out the score as a >value between -1 and 1 that represents the expected outcome. You would scan a >database of games and find that there are say 9000 positions that I evaluate as >0.20, but you would find that the average outcome of those positions is actually >0.35. You would complain that I'm giving you wrong information, and you would be >right in a sense. > >Even if my score would be right for any game database that you can find, it's >not obvious to me that it would still be correct in say a Fritz vs. Junior >match, for many reasons. One is that computers tend to have different positions >than humans, another that Junior has a better understanding of the games it >plays itself, and the fact that one program is stronger than the other would >also skew the "correct" score in some direction. > >Currently program scores don't have much objective meaning, so there is no >problem, and you can't argue that they are wrong except by showing that the >program loses a lot, which is a weak argument, because many programs have wrong >evaluations and win. > >A few months ago I suggested here (or maybe in rgcc I don't remember) to use the >score interpretation as expected outcome as a way of tuning the evaluation >function. The idea was simply that what the program evaluates should match the >actual percentages that occurred in many games. This method of tuning looked to >me very attractive because it uses evaluation only and doesn't depend on search. > >I'm using this method as a rough test if a change I'm doing is in the right >direction. It had some nice successes, and also a few failures. I don't trust it >on its own. A change in formulation may perhaps make it more successful. > >Amir See this quarter's ICCA journal for a discussion of Knightcap's temporal difference learning about this very same subject. -- Komputer Korner
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.