Author: Uri Blass
Date: 11:41:11 01/05/04
Go up one level in this thread
On January 05, 2004 at 13:57:05, Christophe Theron wrote: >On January 05, 2004 at 03:14:48, Uri Blass wrote: > >>On January 04, 2004 at 23:04:55, Christophe Theron wrote: >> >>>On January 04, 2004 at 11:10:41, Bob Durrett wrote: >>> >>>>On January 04, 2004 at 11:00:31, Dan Andersson wrote: >>>> >>>>> I admire your persistance. I guess most of us that have a mathematical >>>>>statistics education got tired explaining things after the first thread or so. >>>>> >>>>>MvH Dan Andersson >>>> >>>>I, too, have a "mathematical statistics education." >>>> >>>>What bugs me is that all of the CCC bulletins seem to suggest that those who run >>>>and evaluate tournaments look only at the win/loss statistics. There is >>>>considerably more information in a game score than just the final game result. >>>> >>>>Throwing away useful information is what I call "blind adherence to statistics." >>>> One needs to rise above one's formal education and supplement it with good >>>>thinking. >>>> >>>>: ) >>>> >>>>Bob D. >>> >>> >>> >>>The games themselves do not contain more information about the relative strength >>>of the opponents than the bare winning percentage of the winner. >>> >>>That should not be forgotten. >>> >>> >>> >>> Christophe >> >>No >> >>The games have more information but the problem is how to interpret them. >>Let give an extreme example when it is easy to learn from only 2 games who is >>better. >> >>Suppose that the loser program in both games does mistakes that 2 ply search can >>avoid and not one mistake. >> >>First it is losing a pawn and later it is losing a knight and later the queen >>and finally it is checkmated. >> >>Suppose that it happens in 2 games. >> >>You can say that the winner is better based only on the games but you cannot say >>it based only on the results. >> >>Usually learning from the games is harder but it does not mean that they have >>not more information then the results. >> >>Uri > > > >No Uri. I could write a program that once in a while, randomly, would throw a >game away by doing only a 2-plies search. Then in the other games it would play >at full strength. > >You would be wrong assuming anything about the program by looking at those 2 >lost games. > >You would still have to play a large number of games to assess its real >strength. > >This example is not unreal anyway: it's what happens when a program has a bug >that make it throw away certain games. It can happen in one game but when it happens in 2 consecutive games then I assume that it is probably not because of a bug. > >The content of the games themselves does not tell you anything more about the >relative playing strength of the opponents than the winning percentage. You can never be 100% sure but the content of the games can change the confidence that you believe in something. > >From a mathematical point of view, it's obvious anyway: the formula to compute >the relative playing strength of two opponents does not take into account the >contents of the games themselves. It just takes into account the number of wins, >draws and losses. Or just the winning percentage. The problem is that it is not simple to have a formula that use the content of the games but it does not mean that the content of the games has no meaning. If you add knowledge about pawn endgames without speed reduction and you see that the program play the same in most positions and better in pawn endgame you can learn that the program is probably better and testing by thousands of games is simply a waste of time. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.