Author: rasjid chan
Date: 04:11:02 04/30/04
Go up one level in this thread
On April 30, 2004 at 06:31:18, Frank Phillips wrote: >On April 30, 2004 at 04:27:23, rasjid chan wrote: > >> >>The Secret of the Commercial Chess Programmers IMNSHO. >> >>Omid mentioned Falcon reached 2600 after improvements in search and >>now he runs out of ideas to push search further and had to try >>to improve on his eval. Again eval would reach a barrier as most >>programs with 2600 has a very good eval and the eval curve again >>shows diminishing returns for code-increase / additional factors. >>The only way is to be able to have a vertical upward displacement >>for this eval-curve which is basically logarithmic - it may be +100 elo >>by just using the correctly tuned weights. >> >>The Deep Blue team mentioned they used MLR and since it can be done, >>it means it should not be too difficult as MLR itself is not very high >>mathematics. I used MLR before and has a not-yet-complete idea of using >>it to tune weights. >> >>The requirement is we need a collection of say 100 drawn game-files, the >>more the bettter as reading and applying eval() at all nodes for a 1000 >>games files may just take a second more. >> >>MLR tuning is logically based and it should not be too bad, at least >>we have some indications, but again we may not know it could be the BEST >>way to do weights tuning. Also it has a very high return if it works. It is >>a one time investment and once the tuner is written and we keep the >>100 game-files, it could be re-used anytime we change our eval(), we only >>need 10 mins to do a rerun and we have all the weights re-tuned and we >>never need to worry too much what weights we assign when we add a >>new factor. >> >>The following is what I arrive at on first thought, it may be triple dumb! >> >>MLR is about finding a set of constants c0,c1,c2... for >> >>Y = c0 + c1*X1 + c2*X2 + ..... >> >>As I have not think it through completely, others may be able to >>complete the picture and let's see if it can be done easily. >> >>MLR requires us to collect a set of table data : >> >>per row: >> >>Y , X1, X2, X3,...., Xn >> >>a row/more is added to the table after applying our tuner to list thru the >>game-files and making the moves and apply score = eval() to the >>"neutral nodes". >> >>scheme 1:- >>Y is the should-be-near-zero score. >>Xn is our current weights used. >> >>scheme 2 - a "dumb" improvement :- >>Instead of using Xn we use del_Xn,the change in Xn alone(when only Xn is allowed >>to change) needed to make score == zero. >>After computing all del_Xn, we use >>Y = score = eval() using Xn + del_Xn instead of Xn. >>This may be better as weights not activated in certain nodes >>will have del_Xn = 0; >>Is this scheme ok ? >> >>Also in general for MLR, Y and Xn in should be scaled using a >>weighted average. Most common weights may vary about a certain mean value. >>Nodes should also be weighted, the nodes that eval() to far-from- >>zero are given less emphasis/weights/rows as the games are supposed to >>be drawn games. >> >>After running MLR, we obtain solution to >>c0,c1,c2... etc and this enables us to compute the optimal change >>needed to our weights. The correlation coefficient R2 from MLR may indicate >>how reliable the proposed changes is, or it may throw some lights >>on the quality of our eval(), etc. >> >>Also recording how often a factor is activated on average can let us decide >>if it may be discarded or used with a more conservative weight. >>MLR may also give us other indications when we may know certain factors >>are not important/or unstable when the tuned weights indicate an >>exceptionally high change needed, ie the weight or the factor itself may be >>suspect and should to be discarded. >> >>Should I do it now when my program is at 1900 ? >> >>Rasjid > >Just some comments, rathr than an answer to your question. > >I would expect a program with a standard search (pvs + nullmove + hash tables) + >any sensible evaluation running a modern cpu to be above 1900. Well above 1900. > >One observation from experience with my own program (strictly second division) >is that its average performance is relatively resistant to changes to the evaluation. This is interesting. Maybe it relates to a general principle about writing an eval() or identifying some critical factors that is not known yet to mere chess programmers. I did some crude test in the past adding more to TSCP, N/B passed-pawn with pawn_prot,lone bishop considerations and bishop pairs and maybe a little more. Seems the same. But still as others posted, and from theory of alpha-beta that it relies only on one signal, ie eval(), evaluation determines how alpha-beta searches and the returned best-move. This may indicate that most of the stuff I keep adding is marginal >at best ie it contributes little overall, although sometimes is critical in >specific situations. A few well tuned key evaluation factors and a fast search >should take you a very long way .... at least against other computers and less >than IM level humans. > >Deep Thought used statistical regression to generate weighting factors. You can >still download the code from the web. I though Deep Blue also benefited from GM >hand-tuning as well. There is mentioned Deep Blue uses GM games and MLR. Rasjid
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.