Author: rasjid chan
Date: 08:18:27 04/30/04
Go up one level in this thread
On April 30, 2004 at 08:54:55, Anthony Cozzie wrote: >On April 30, 2004 at 07:26:55, rasjid chan wrote: > >>On April 30, 2004 at 06:59:15, Uri Blass wrote: >> >>>On April 30, 2004 at 04:27:23, rasjid chan wrote: >>> >>>> >>>>The Secret of the Commercial Chess Programmers IMNSHO. >>>> >>>>Omid mentioned Falcon reached 2600 after improvements in search and >>>>now he runs out of ideas to push search further and had to try >>>>to improve on his eval. Again eval would reach a barrier as most >>>>programs with 2600 has a very good eval and the eval curve again >>>>shows diminishing returns for code-increase / additional factors. >>>>The only way is to be able to have a vertical upward displacement >>>>for this eval-curve which is basically logarithmic - it may be +100 elo >>>>by just using the correctly tuned weights. >>>> >>>>The Deep Blue team mentioned they used MLR and since it can be done, >>>>it means it should not be too difficult as MLR itself is not very high >>>>mathematics. I used MLR before and has a not-yet-complete idea of using >>>>it to tune weights. >>>> >>>>The requirement is we need a collection of say 100 drawn game-files, the >>>>more the bettter as reading and applying eval() at all nodes for a 1000 >>>>games files may just take a second more. >>>> >>>>MLR tuning is logically based and it should not be too bad, at least >>>>we have some indications, but again we may not know it could be the BEST >>>>way to do weights tuning. Also it has a very high return if it works. It is >>>>a one time investment and once the tuner is written and we keep the >>>>100 game-files, it could be re-used anytime we change our eval(), we only >>>>need 10 mins to do a rerun and we have all the weights re-tuned and we >>>>never need to worry too much what weights we assign when we add a >>>>new factor. >>>> >>>>The following is what I arrive at on first thought, it may be triple dumb! >>>> >>>>MLR is about finding a set of constants c0,c1,c2... for >>>> >>>>Y = c0 + c1*X1 + c2*X2 + ..... >>>> >>>>As I have not think it through completely, others may be able to >>>>complete the picture and let's see if it can be done easily. >>>> >>>>MLR requires us to collect a set of table data : >>>> >>>>per row: >>>> >>>>Y , X1, X2, X3,...., Xn >>>> >>>>a row/more is added to the table after applying our tuner to list thru the >>>>game-files and making the moves and apply score = eval() to the >>>>"neutral nodes". >>>> >>>>scheme 1:- >>>>Y is the should-be-near-zero score. >>>>Xn is our current weights used. >>>> >>>>scheme 2 - a "dumb" improvement :- >>>>Instead of using Xn we use del_Xn,the change in Xn alone(when only Xn is allowed >>>>to change) needed to make score == zero. >>>>After computing all del_Xn, we use >>>>Y = score = eval() using Xn + del_Xn instead of Xn. >>>>This may be better as weights not activated in certain nodes >>>>will have del_Xn = 0; >>>>Is this scheme ok ? >>>> >>>>Also in general for MLR, Y and Xn in should be scaled using a >>>>weighted average. Most common weights may vary about a certain mean value. >>>>Nodes should also be weighted, the nodes that eval() to far-from- >>>>zero are given less emphasis/weights/rows as the games are supposed to >>>>be drawn games. >>>> >>>>After running MLR, we obtain solution to >>>>c0,c1,c2... etc and this enables us to compute the optimal change >>>>needed to our weights. The correlation coefficient R2 from MLR may indicate >>>>how reliable the proposed changes is, or it may throw some lights >>>>on the quality of our eval(), etc. >>>> >>>>Also recording how often a factor is activated on average can let us decide >>>>if it may be discarded or used with a more conservative weight. >>>>MLR may also give us other indications when we may know certain factors >>>>are not important/or unstable when the tuned weights indicate an >>>>exceptionally high change needed, ie the weight or the factor itself may be >>>>suspect and should to be discarded. >>>> >>>>Should I do it now when my program is at 1900 ? >>>> >>>>Rasjid >>> >>>No >>> >>>I believe that Crafty never used MPL and it is clearly better than 1900. >>> >> >> >>>I also believe that the secret of Shredder is mainly better search than Crafty >>>and not better evaluation(I do not say that it does not have better evaluation >>>but I guess that most of the rating difference between Crafty and Shredder is >>>because of the fact that Shredder has a better search). >> >>Not much dispute. But we know one thing. After a commercial chess program >>reaches ELO 2600+, you either tell the author he's redundant or we can >>imagine he goes all way to to seek even little improvements... like someone >>testing with 2/3 computers running for days/weeks(months ???). >> >>Top programs must be top in BOTH SEARCH AND EVALUATION - no short cut >>as none have suggested that one can completely replace the need for the >>other, or even that we can have one very good and the other plain and >>simple. >> >>But if my guess is correct, and if MLR tuning works good, then it is >>HIGHLY BENEFICIAL as a one time investment. Commercial chess programmers >>have been on their programs for years and my guess is they should have >>tried MLR tuning, at least to see if it is reliable. >> >>I see MLR tuning may not be difficult and I definetly would spend a few >>weeks to test if my program ever reach ELO2500 >> >>Rasjid > >Don't feel bad about Uri. He begins all his posts with "no" :) > >BTW, if your program is rated 1900, you almost certainly have bugs. IMO >2200-2300 ICC rating is the "more-or-less-bugfree" zone. That is with a basic >eval, searching 9-10 ply with null move, PVS, killers, history, transposition >table, etc. > Currently my progam cannot be rated. After I joined this board, and after reading Ed's article, I think I now know there may be ways to write a reasonable program. I have stripped down my eval() so that it does not interfere with other things. Rasjid >anthony > >>> >>> >>>Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.