Author: Peter Berger
Date: 16:10:39 01/28/01
Go up one level in this thread
On January 28, 2001 at 17:00:30, Severi Salminen wrote: >>It is very strange. Chess programs are all using, more or less, the same basic >>principles. So the logical way is to assume that they all benefit more or less >>equally from faster hardware. But I have never seen anybody supporting this >>assumption. Instead of trying to demonstrate that this simple assumption is >>wrong, everybody just assumes that it is wrong. Why? >> >>I guess the answer is that it is more fun to assume that all chess programs do >>not benefit from faster hardware in the same way. So people believe that by >>changing the hardware or the time controls big surprises can happen... >> >>On the other hand it is always hard to explain that in short matches big >>surprises can happen FOR NO REASON. >> >>So people tend to draw flawed conclusions based mainly on their beliefs, and to >>present them as scientifical evidence... > >This all is result of human nature. We want to understand things we don't >understand. We want to create our own set of rules in order to forecast complex >systems. Same in computer chess: people love to see different characteristics in >different programs (Gambit Tiger is a brave attacker, Hiarcs plays positional >chess, Fritz tactical...). They want to see these "new paradigms" and want to >categorize programs' behaviour based on a few games. They want to see a >human-like behaviour. And it also looks like the people who make these >conclusions are usually not programmers (IMO :). And I don't blame them. It is >impossible to know how chess engines _really_ function unless you have tried it >out yourself. And for marketing point of view it would be quite boring if all >engines were presented as little modifications of same principles that have been >around 30 years, wouldn't it. I wouldn't be suprised if Fritz and Junior were >actually the same engine :) > >The point: let them have their paradigms and let us have our scientifical facts. >We can filter the useful inforamtion out. In this case maybe 500 games could not >be enough to show anything - if there is anything to show. > >Severi I tend to believe statistically significant results are overestimated : they are so easy to get : only takes _time_ : oops , might this be the reason they are that rare ? Look at Mr Heinz' results for the decreasing one-more-ply-effect : from a statistical point of view it is quite easy to question his results and require even more experiments to eliminate the "noise" , isn't it ? I suspect it is quite easy to prove that certain programs profit more from better hardware than others : these Nimzo tests are a good start btw : to question the reliability of these results is perfectly OK for sure : but they point into a certain direction ; statistics is simple and difficult at the same time ; what some people seem to forget : even if you play a too little number of games you can place a bet which is better than 50 % , a thing people do all day IRL ; I suspect with this Nimzo data we are already way over 60 % btw ; might still be all nonsense for sure ... The tools are there and it is tempting to simply do it to end this "battle" . To avoid the question " Is it better hardware or does program X simply suck at blitz" it is probably better to choose fast time control , then something like ERT , 500 games each , time control maybe 5 minutes game /3 secs increment ; opponents maybe a Tiger or Crafty against a Gandalf or a Nimzo on a fast and a slow compi ; but statistics is tricky , else this would probably already simply have been done ; questioning a result created this way is pretty easy , creating it pretty time-consuming IMHO . pete
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.