Author: Howard Exner
Date: 22:34:40 03/19/98
Go up one level in this thread
On March 19, 1998 at 17:13:10, Mark Young wrote: >On March 19, 1998 at 14:36:43, Howard Exner wrote: > >>On March 19, 1998 at 13:03:11, Sylvain Renard wrote: >> >>> >>>On March 15, 1998 at 22:50:22, Mark Young wrote: >>> >>>>The Match will end when one program wins 10 games draws not counting. >>>>If the score is 9-9 the match will end as draw. >>>I think that the number of games played will be too low to have a >>>scientific >>>result. >>>I have started a match between the Fritz4.01 and Fritz 5 engines, >>>same opening books , PP 180 MHz , 32 MB of hash tables for each >>>program, "blitz" 1 hour. >>> After 40 games, there score was 20 - 20. >>> But after 94 games Fritz 4.01: 42,5 Fritz 5: 51,5 (20 - 29 - 43 >>>draws). >> >>How many double games occured in this 94 game match? >> >>>Conclusion: when the ratings of the programs are close, you have to >>>play many many games before you can say that one is SLIGTHLY better >>>that the other. As I think that ALL programs ahead in the Sweedish >>>Rating >>>List are close, THOUSANDS of games are necessary and any conclusion >>>based on 1 game (I have read some of them here...), 11 games or even >>>100 games is RIDICULOUS. >> >>I agree that a large sample size is needed for comp vs comp games. If >>you >>played another 94 games the match could very well be tied again. > > > >I strongly reject the logic that match results and game results are >meaningless. >If this was the case you can only go by your feelings what program is >better. I'm referring to the notion on drawing conclusions from a single match between two computer opponents - nothing more. I'm not sure, but are you saying that if A beats B then you could conclude that A is the stronger opponent? However, if you believe that more matches are required against other opponents before concluding which program is stronger then I am in agreement with you. That would be another topic than the above. I have been testing computer programs for years and found over >the years that the cream does rise to the top. I am following your match with great interest but will not draw any conclusions from the match result.(other than the obvious conclusion that A beat B). I will however draw conclusions from playing over the games. Thanks for posting them.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.