Author: Eric Oldre
Date: 22:34:48 07/09/04
Go up one level in this thread
On July 10, 2004 at 01:02:39, Russell Reagan wrote: >Okay, I'll try again :) > >Allowing people to bring any hardware they want is the fairest way to run a >computer chess tournament. Any attempt to equalize the hardware will cause more >problems than it solves. > >Point 1: Not possible > >Accomplishing a fair, equal hardware event is not possible. No matter what you >do, you penalize someone arbitrarily. Everyone chooses different areas to focus >on, and the tournament really becomes unfair when you penalize someone heavily >for an arbitrary reason. This is much more unfair than allowing different >hardware. Open hardware doesn't really affect the outcome drastically, for >reasons described below. > >Point 2: Not needed > >There is no need for an equal hardware computer chess tournament. Searching a >chess game tree is an exponential problem. Unless the engines are very close in >playing strength, hardware isn't going to affect the outcome. > >In the early 1990s PC programs started winning the WCCC, despite playing on much >slower hardware than the hardware projects like HiTech and Deep Blue >(prototype). > >Crafty and Junior played a handicap match where Crafty had a 10:1 time >advantage. Crafty won, barely. > >Diep ran on a super computer with 512 processors in the 2003 WCCC. It finished >6th. > >In all three of these situations (Deep Blue prototype, Crafty, and Diep) the >better software won despite being at a hardware disadvantage. > >It has been a long time since hardware stole a WCCC. There really isn't any need >to equalize hardware, and there are certainly huge downsides to trying. > >Point 3: No benefit > >There is no added benefit of an equal hardware event. People say that they want >an equal hardware event so that they know who has the best software, usually so >they know what will run best on their home computer. There are several problems >with this. > >First, there are not enough games to really decide which software is the best. >You need many more games between two opponents to get an idea who is better. >Even a weak engine will get a draw against a strong one from time to time. > >Second, the result of the tournament is sometimes heavily influenced by some >freak occurance. For instance, Fritz locked up against Falcon and lost on time >this year at the 2004 WCCC. Currently Fritz is 1 behind the leader. That could >be the difference in the outcome of the event right there. A tournament is not >even close to a scientific experiment. The SSDF is the closest thing you can >find to that, and it is better than any tournament in that respect by a mile. > >Third, if the equal hardware is AMD Opteron machines, that doesn't tell you what >will perform the best on your Pentium 4 machine at home, or your friend's 32-bit >Athlon XP. > >So you don't learn anything more from a 10 round equal hardware tournament than >you do from a 10 round open hardware tournament. Russell, I agree with you for the most part. hardware will only make a difference with two engines that are fairly evenly matched. for any competition to be fair the programs must be allowed to run on the type of hardware that the author has targeted. authors who target less popular platforms should not be excluded from the event. and using identicle hardware will eliminate some engines, and penalize others. I personally think an event of this type would be fun with or without equal hardware, but many people have posted concerns over differences in hardware. my perception of their concerns is that people don't object to different architectures, but feel that someone with access to a $30,000 server has a distinct advantage, even if it only will come into play against similar strength engines. I think that putting a $ limit on the hardware used does a decent job of addressing those concerns while still allowing authors to use the platform they targeted in development. i am NOT suggesting that the WCCC use this type of scheme, but if the one of the goals for a new competition was to get as many people to show up as possible for a over the board match. then a rule like this might get some of those who are scared off by huge hardware differences to participate. Eric
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.