Author: Mark Young
Date: 23:15:05 01/03/04
Go up one level in this thread
On January 04, 2004 at 00:42:02, Christophe Theron wrote: >On January 03, 2004 at 20:53:39, Rick Rice wrote: > >>Person A posts a message saying Ruffian 2.0 is very dissapointing, with the >>results to back it up. This is followed by a second post which basically says >>that Ruffian 2.0 rocks with some results to back it up. Are these programs >>really so time and hardware sensitive, so as to show varying results on >>different CPUs/time controls? >> >>Ideal solution would be for SSDF to have one massive board with one CPU and >>memory for each program (equal CPU and mem for all the progs on its list) and >>some way to automate the play of these programs against each other..... on >>different time controls such as regular, blitz etc. Just wishful thinking for >>the future, but it would eliminate the multiple and varying results. >> >>Cheers, >>Rick > > > >Statistics are extremely important in chess, and in computer chess. > >Unfortunately, even after years of talks about the subject, almost nobody on >this message forum understands that you really need A LOT OF GAMES to start to >have an impression of a probability about which program is stronger. You need to be more accurate. This is not always true. You need more games the closer the two programs are in strength. A 20 game match is more then enough if you score something like 17 - 3 or better. If so there is a very high likely hood that the winner is stronger. I did not have to play 10,000 games to know Shredder was stronger then bam bam. > >The variations you have noticed do not come from different setups. > >These variations are statistical variations. That means that most of the match >results posted here are statistically MEANINGLESS. > >People love to proudly post the result of the 20 games match they have run >overnight. They don't even care to know if that result has any meaning. Well in >most of the cases the result means nothing (just a waste of electric power) and >you should not care about it at all. I don't think people posting on ICC is a waste of electric power. It is one piece of data. If that is all you had... It would not mean much, but when many post 20 game match results. It can mean something. > > > > Christophe
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.