Author: Tim Foden
Date: 09:47:23 04/06/04
Go up one level in this thread
On April 06, 2004 at 09:20:02, Geert van der Wulp wrote: >On April 05, 2004 at 10:51:06, Tim Foden wrote: > >>On April 05, 2004 at 08:26:08, Andrew Wagner wrote: >> >>> >>>I'm taking this intro to calc-based statistics class this semester, and it got >>>me thinking. We tend to generate a lot of statistics during a game. In fact, we >>>could generate a LOT more. But we do very simple things with these statistics. >>>For example, we might add some bell or whistle to the search, notice that it >>>doesn't show any obvious increase in NPS or move ordering, or whatever we were >>>hoping for, and ditch it. That's an extremely simple analysis of the statistics. >>>I'm wondering if anybody has done any more advanced statistical work in figuring >>>out what factors are important to winning a game. Here are two examples I >>>thought of: >>> >>> 1.) Using advanced statistics to calculate better values to be used in >>>static eval. What if we took a ton of super-gm games, and generated a slew of >>>statistics about each game like number of moves white has an isolated pawn on, >>>or number of moves black keeps his king-knight, and so on. Then do a >>>multi-variate analysis to determine whether any of those things have an effect >>>on the probability that white or black will win. Unfortunately, this math is a >>>little above me at this point, and maybe someone else has tried it, I don't >>>know. It's an interesting idea to think about, though. >> >>Dave Gomboc has a thesis paper (Ordinal Correlation for Evaluation Function >>Development) on generating evaluation values from annotated games. This may be >>of interest. >> >>I've used multi-variate linear regression to generate a set of evaluation >>values, but I have some bugs in the implementation (as pointed out by Dave >>Gomboc when I spoke to him in Graz), and I haven't tested the values at all. I >>still plan to pursue this some more when I have the time and the inclination. >> >>For the sake of interest, I've added below one of the eval result sets that were >>produced. The 4 different colums are progressing (left to right) from the >>opening to the endgame (based on the material value of the pieces, not including >>pawns). If anyone wants some more info on the weights or anything, please feel >>free to ask. :) >> >>Cheers, Tim. >> >>============================================================= >> >>BEGIN weights >> "pawn material value" 1.1406 0.90261 1.0566 1.2131 >> "knight material value" 3.567 3.1712 3.1393 3.0996 >> "bishop material value" 4.1172 3.533 3.3734 3.2835 >> "rook material value" 6.0329 4.813 4.9644 5.2757 >> "queen material value" 7.1422 9.5802 9.4663 9.1281 >> "attack knight -> king" 0.30462 0.24993 0.12274 0.013293 >> "attack bishop -> king" 0.13924 0.054991 -0.0021642 -0.04052 >> "attack rook -> king" -0.0017353 0.045746 0.044572 0.022176 >> "attack queen -> king" 0.10511 0.22902 0.26486 0.14355 >> "attack knight orth -> king" 0.047577 0.025824 -0.01211 -0.020798 >> "attack bishop orth -> king" -0.025117 0.0049062 -0.014941 -0.031793 >> "attack rook orth -> king" 0.10143 0.05584 0.0074275 -0.003275 >> "attack queen orth -> king" -0.017915 0.017174 0.034982 0.031466 >> "pawn centre control" -0.0019067 -0.082808 -0.17489 -0.13969 >> "pawn islands" 0.62435 0.26761 0.085621 0.0033765 >> "pawn passed 7th" 2.2623 1.7357 1.6928 1.5856 >> "pawn passed 6th" 2.0762 1.3918 1.1353 1.0766 >> "pawn passed 5th" 1.4342 0.7854 0.65716 0.64748 >> "pawn passed 4th" 0.7573 0.32951 0.27647 0.30201 >> "pawn passed 3rd" 0.2687 0.051558 0.0040888 -0.017349 >> "pawn passed 2nd" 0.15252 0.096773 -0.02398 -0.060491 >> "pawn potentially passed 6th" 2.0946 1.5579 1.0143 1.0635 >> "pawn potentially passed 5th" 1.2708 0.81233 0.57991 0.48609 >> "pawn potentially passed 4th" 0.50466 0.31012 0.2499 0.24497 >> "pawn potentially passed 3rd" 0.22033 0.087207 0.064912 0.068724 >> "pawn potentially passed 2nd" -0.056561 -0.061395 -0.072704 -0.067356 >> "pawn not passed 6th" 1.1617 0.73218 0.50662 0.4044 >> "pawn not passed 5th" 0.73137 0.2938 0.18061 0.13773 >> "pawn not passed 4th" 0.33807 0.091941 0.021417 -0.027793 >> "pawn not passed 3rd" 0.15847 -0.01997 -0.070619 -0.11419 >> "pawn structure isolated" -0.036192 -0.020619 -0.08264 -0.12495 >> "pawn structure protected" 0.22706 0.30687 0.19189 0.080273 >> "pawn structure duo" 0.13775 0.1228 0.084101 0.064057 >> "pawn structure doubled" -0.35537 -0.055624 -0.13756 -0.26692 >> "pawn king shelter A-L2" -0.022126 0.005001 -0.20839 -0.54978 >> "pawn king shelter A-L3" 0.076894 0.13139 -0.050282 -0.39465 >> "pawn king shelter A-L4" -0.11582 0.028443 -0.10683 -0.40383 >> "pawn king shelter A-M2" -0.022673 0.13684 0.14164 -0.079003 >> "pawn king shelter A-M3" -0.16068 0.032049 0.010325 -0.24427 >> "pawn king shelter A-M4" -0.27629 -0.15283 -0.16818 -0.39492 >> "pawn king shelter A-R2" -0.11039 0.11641 0.18556 0.22237 >> "pawn king shelter A-R3" -0.16935 -0.0040184 0.014824 -0.029603 >> "pawn king shelter A-R4" -0.065908 0.037064 -0.050528 -0.24079 >> "pawn king shelter B-L2" -0.080971 0.0038952 -0.14428 -0.52359 >> "pawn king shelter B-L3" -0.10675 0.0089823 -0.092122 -0.3781 >> "pawn king shelter B-L4" -0.13398 -0.019871 -0.1148 -0.32942 >> "pawn king shelter B-M2" 0.20159 0.26134 0.10625 -0.24626 >> "pawn king shelter B-M3" 0.2194 0.28104 0.18137 -0.14319 >> "pawn king shelter B-M4" -0.022874 0.10666 0.065321 -0.27403 >> "pawn king shelter B-R2" 0.17474 0.25429 0.24577 0.17234 >> "pawn king shelter B-R3" -0.097654 0.067401 0.10321 0.010602 >> "pawn king shelter B-R4" -0.096961 0.016401 -0.0049095 -0.17759 >> "pawn king shelter C-L2" 0.056486 0.13152 -0.021853 -0.38982 >> "pawn king shelter C-L3" -0.26401 0.033654 -0.074541 -0.43137 >> "pawn king shelter C-L4" -0.40364 0.059407 0.016478 -0.24902 >> "pawn king shelter C-M2" 0.023612 0.15191 0.10286 -0.1438 >> "pawn king shelter C-M3" -0.27187 0.14494 0.13484 -0.17733 >> "pawn king shelter C-M4" -0.054458 0.059772 0.064958 -0.25615 >> "pawn king shelter C-R2" -0.083708 0.16986 0.24501 0.078153 >> "pawn king shelter C-R3" -0.29241 -0.015877 0.056394 -0.056775 >> "pawn king shelter C-R4" -0.032473 0.0027616 0.043754 -0.15224 >> "pawn king shelter D-L2" -0.24029 0.087717 0.01187 -0.24422 >> "pawn king shelter D-L3" -0.53426 0.03321 0.0015179 -0.25848 >> "pawn king shelter D-L4" -0.34867 0.0069573 0.031903 -0.34863 >> "pawn king shelter D-M2" -0.42294 -0.044203 0.1078 -0.032897 >> "pawn king shelter D-M3" -0.057296 0.18677 0.15735 -0.16693 >> "pawn king shelter D-M4" -0.09145 0.18919 0.2055 -0.23548 >> "pawn king shelter D-R2" -0.44007 -0.022726 0.092448 -0.084192 >> "pawn king shelter D-R3" -0.27404 0.041379 0.050231 -0.20311 >> "pawn king shelter D-R4" -0.21693 -0.011397 -0.016771 -0.22884 >> "knight centralization" 0.14191 0.20221 0.29341 0.42521 >> "knight mobility 0" -0.83457 -0.62452 -0.58155 -0.9075 >> "knight mobility 1" -0.26076 -0.42072 -0.38269 -0.38084 >> "knight mobility 2" -0.056587 -0.27456 -0.23687 -0.25139 >> "knight mobility 3" -0.039265 -0.14174 -0.12473 -0.13854 >> "knight mobility 5" -0.021017 0.039356 0.060901 0.064418 >> "knight mobility 6" -0.067184 0.032043 0.053965 0.023695 >> "knight mobility 7" -0.012212 0.044088 0.025386 -0.06144 >> "knight mobility 8" 0.097228 0.019449 -0.071263 -0.18899 >> "bishop centralization" -0.1305 0.11118 0.19771 0.18006 >> "bishop pair" 0.9769 0.82948 0.93598 1.0589 >> "bishop trapped" -1.7152 -0.72841 -0.64914 -0.57104 >> "bishop mobility 0" -0.097897 -0.75452 -0.60998 -0.75167 >> "bishop mobility 1" -0.44464 -0.76651 -0.67837 -0.64148 >> "bishop mobility 2" -0.23056 -0.55471 -0.53568 -0.59763 >> "bishop mobility 3" -0.19882 -0.36018 -0.37739 -0.41701 >> "bishop mobility 4" -0.06746 -0.19106 -0.22049 -0.24238 >> "bishop mobility 5" -0.0091593 -0.096794 -0.099845 -0.10833 >> "bishop mobility 7" 0.06958 0.028875 0.01518 0.022421 >> "bishop mobility 8" 0.072722 0.085421 0.10151 0.12959 >> "bishop mobility 9" 0.082447 0.071574 0.08158 0.11276 >> "bishop mobility 10" 0.12631 0.076462 0.097483 0.12912 >> "bishop mobility 11" 0.35584 0.1593 0.08263 0.072669 >> "bishop mobility 12" 0.41713 0.42934 0.18616 0.1527 >> "bishop mobility 13" 0.84487 0.5812 0.062758 -0.0027475 >> "rook doubled" 0.70149 0.33454 0.1719 -0.25325 >> "rook on open file" 0.46004 0.2933 0.17586 0.089891 >> "rook on 7th with king" 0.31016 0.32808 0.58025 0.60076 >> "rook on 8th with king" 0.55981 0.30994 0.39403 0.36868 >> "rook mobility 0" -0.27746 -0.43129 -0.50345 -0.72407 >> "rook mobility 1" -0.5577 -0.33474 -0.42349 -0.62703 >> "rook mobility 2" -0.4703 -0.26137 -0.32301 -0.4228 >> "rook mobility 3" -0.35126 -0.20671 -0.23946 -0.21282 >> "rook mobility 4" -0.30383 -0.18541 -0.16228 -0.15732 >> "rook mobility 5" -0.14291 -0.076933 -0.063457 -0.093823 >> "rook mobility 7" 0.17168 0.077755 0.069203 0.047249 >> "rook mobility 8" 0.30126 0.20285 0.15249 0.10609 >> "rook mobility 9" 0.326 0.24026 0.20956 0.15862 >> "rook mobility 10" 0.32201 0.28211 0.25892 0.1893 >> "rook mobility 11" 0.30765 0.30691 0.29464 0.20213 >> "rook mobility 12" 0.52154 0.38927 0.33822 0.21387 >> "rook mobility 13" 0.31582 0.31439 0.31284 0.19524 >> "rook mobility 14" 0.79975 0.41196 0.2966 0.17407 >> "queen mobility 0" -0.47345 -0.71896 -0.65612 1.6311 >> "queen mobility 1" -0.40729 -0.63267 -1.2863 -3.2576 >> "queen mobility 2" -0.33689 -0.47483 -0.66575 -1.4982 >> "queen mobility 3" -0.21585 -0.43171 -0.58621 -0.018905 >> "queen mobility 4" -0.32783 -0.37993 -0.39452 -0.15295 >> "queen mobility 5" -0.20645 -0.27481 -0.32489 -0.095064 >> "queen mobility 6" -0.29378 -0.21198 -0.27622 -0.12774 >> "queen mobility 7" -0.15181 -0.15694 -0.22016 -0.051221 >> "queen mobility 8" -0.11013 -0.098562 -0.13181 -0.12578 >> "queen mobility 9" -0.058923 -0.042352 -0.05672 -0.11408 >> "queen mobility 11" 0.049511 0.047283 0.048802 -0.035158 >> "queen mobility 12" 0.067134 0.083217 0.060141 -0.023557 >> "queen mobility 13" 0.040709 0.11823 0.13458 0.036909 >> "queen mobility 14" -0.011342 0.14935 0.14643 -0.062753 >> "queen mobility 15" 0.053293 0.17975 0.17374 0.0023686 >> "queen mobility 16" 0.073484 0.19323 0.18902 -0.0027121 >> "queen mobility 17" 0.11043 0.23795 0.20996 0.037198 >> "queen mobility 18" -0.20202 0.26655 0.22943 0.036928 >> "queen mobility 19" -0.11242 0.29996 0.23644 0.0088159 >> "queen mobility 20" 0.36078 0.37786 0.23989 -0.044868 >> "queen mobility 21" 0.14893 0.44521 0.22468 -0.19958 >> "queen mobility 22" 0.59181 0.5038 0.24374 -0.043637 >> "queen mobility 23" 0.88926 0.56728 0.20571 -0.14894 >> "queen mobility 24" 0.81131 0.47311 0.19477 -0.042795 >> "queen mobility 25" -0.87389 0.51089 0.052824 -0.30345 >> "queen mobility 26" 6.1497 1.0237 0.32422 0.02058 >> "queen mobility 27" 2.8812 0.51972 0.18707 -0.16137 >>END weights >> >> Number of observations (N): 17289026 8346403 8048301 4998813 >> Sum of squares error (SSE): 1.113266e+007 4895954 3750259 1818094 >> Mean square error (MSE): 0.6439201 0.5866044 0.4659772 0.3637155 >> Number of features (n): 140 > >Hi Tim, > >Three questions. In theory of course, the number of "features" is only a choice. >How did you choose them? The program I used is quite flexible, and I can use the features I choose quite easily. This particular run the just the last one I had done in a series, and I had tried to pick features that (1) I felt were important, and (2) that reduced the MSE value when added. >Also it seems obvious that one needs a HUGE number of >games for this. This was run over a database of about 450000 games that is based on the games that used to be available from the the SCID homepage. > Because if you look at your results, it is strange to see the >"queen mobility 21" have a lower value than "queen mobility 15" in the far >endgame. One possibility to explain this is that it is better to be attacking another piece, and thus have slightly lower mobility that you would if you were not attacking any pieces. I agree though that it may be suspect. There are fewer datapoints for the endgame, which will lower the accuracy of the values. >Also I wonder HOW you determine where exactly should be the separation between >opening, middle game and endgame. You say it is based on the total piece value >on the board, but maybe using 4 different clasifications is too much for the >result. I felt that only 1 value was too little, and that there are real differences between the values of features in the opening to those in the ending. I guess 3 would be a minimum that I would consider, but I felt 4 would also be OK. Ranges of total piece values used for categories: Stage 1: 48-99 (opening) Stage 2: 28-56 Stage 3: 12-40 Stage 4: 0-20 >Also I wonder how you check your games for blunders. I ignore them. As I said I make no claims that the values are accurate. :) I don't use positions where the side to move is in check, or where the preceding or following moves are captures. Any other suggestions in this regard are welcome. Cheers, Tim.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.