Author: Eelco de Groot
Date: 15:53:46 08/16/01
Go up one level in this thread
On August 16, 2001 at 16:31:46, Dann Corbit wrote: >On August 16, 2001 at 16:01:11, Nino wrote: >[snip] >>>People get bored of the project and drop off. We are addressing this situation, >>>and there should be something fairly revolutionary to announce soon. We will >>>announce it when it is ready. >> >>Dan can you indicate about when to expect this announcement. > >Don't know for sure, sometime soon. > >>Also dan how much >>experimentation have you done with these rotations? > >Maybe 10 hours total. I have not discovered any problems in my searches, but I >did see a few strange things. For instance, the rotated solution will sometimes >be different than the one that a chess engine will find when left on its own to >analyze the rotated position. However, if you analyze the rotated solution, it >will have about the same score as the original problem. It seems that chess >engine evaluations are not fully symetric for the most part. > >>Has anyone else taken advantage of this concept? > >I doubt it. >[snip] Nino maybe you could ask Harald Faber who also did some of the well-known tests reversed to see, amongst other things, if programs would do worse for example because they were tuned to known testpositions. But he got surprisingly different results too sometimes. Harald e-mailed me some files back then and looking at the results we thought it may have been because of asymmetries. Stefan Meyer-Kahlen had just won yet another of his world-titles and Harald asked him for his opinion about the strange time-differences, Stefan said about the same thing about asymmetries. An example might be if a program always starts looking in the same corner of the board when it generates its movelists, that would be enough to create differences. Once such differences are just enough to cause a good move to be found one ply later you already have a big difference in solution time. It's a bit of chaos theory too I thought; where small differences can grow larger if a program prunes, cuts off, a part of the tree containing a good move that would have been found with a little different move-ordering. Like the butterfly creating climate changes or something like that.. If anybody already has these kinds of results, I think there is some more that you can do with it. Perhaps Les Fernandez thought about something like this too? If you have the results for mirrored positions as well then with a permutation program you can generate all the virtual results AAAAABAAA, BBBAABBBA etc. And with an "off the shelve" calculator suddenly for every testresult you can also generate statistical error-margins, histograms, standard deviations and such. For an experimental program undergoing testing you could look for "outliers": in those positions having very different time-results when mirrored there might also be something wrong in the program. A bug could be causing a systematic rather than a statistical aberration. But, because of the "butterfly-effects", it might not always be that simple to find actual causes and bugs? Big error margins also can tell you that because of random fluctuations you just would have to do more actual testing to get more accurate results. All this more or less in theory, I can Christophe Theron already hear saying that you would have to test this, because over 90% of such ideas just don't work in practice as you had anticipated and mirrored BS-tests just give you "bovine statistics"?! Eelco
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.