Author: Don Dailey
Date: 11:26:48 05/19/98
Go up one level in this thread
On May 19, 1998 at 09:53:48, Ed Schröder wrote: >Hi all, > >Below are some snips from my home page. Hope you like the idea. The >full story, rankings, games, downloading of the games can be found on: > >http://www.rebel.nl/cclintro.htm > >I expect that next monday there will be new results and updated >rankings. > >- Ed Schroder - > >---------------------------------------------- > >This is a new initiative to make computer chess even more attractive. > >Goal is to play thousands of tournament games between the best chess >programs in the world. The chosen form is similar to a normal club >chess competition. > >The competition will be called the COMPUTER CHESS LEAGUE or simply CCL. > >For the moment we have chosen for the 6 leading programs on the famous >SSDF list to participate in the CCL competition. Later (after the first >two rounds and 1000 games are finished) we will include more chess >programs in the CCL competition. For the moment the participating >programs are: > >Mchess 7.1 >Shredder 2.0 >Hiarcs 6.0 >Nimzo 98 >Genius 5.0 >Rebel 9.0 > >All these programs support the auto232 protocol which enables to play >automatic games 24 hours a day, 7 days a week, all fully automatic. > >For the moment 4 identical Pentium-II 266 Mhz machines are used with >64 Mb to play all the games. It's planned later to play the competition >on 6 identical AMD 300 Mhz machines with 64 Mb or 6 identical Intel 400 >Mhz machines. > >Results and new updated rankings will be reported weekly. > >All games played are free downloadable in 4 popular formats (DAT, >CBH, CBF and PGN). Each program will play 2 double rounds (with >white and black). So in total each participant will play 5x2x10 >= 100 games. With 6 participants in total 500 tournament games per >round will be played. With 2 rounds this makes a total of 1000 games. > >After the first two rounds we like to extend the CCL competition >with 4 other strong chess programs which are also auto232 compatible >so in total 10 participants. > >Programmers are welcome to join. With 10 participants we will continue >to play another 2 double rounds (round 3 and 4). From that moment on a >CCL round will take 900 tournament games. With 2 rounds this makes a >total of 1800 games before the winner is known. > >Rules are similar to a normal chess club competition, for example: > >Each match contains 10 games, the winner gets 2 points, a 5-5 result >scores 1 point, a loss of the match 0 points. The game scores will be >counted as the board points similar in chess club competition and will >be used to create an elo rating. > > -------------------- > >Q: What about equal hardware? > >A: All programs will always play on EQUAL hardware on 100% identical >Pc's. There will no speed differences and no differences in memory >size. This to ensure a 100% fair competition. > >Q: What about good hardware? > >A: We strive to play the CCL competition on the best available hardware >currently available. The very first games (50 in total) were played on >4 identical Pentium PRO 200 Mhz machines (of our own) with 32 Mb. > >Now matches are played on 4 identical Pentium-II 266 Mhz machines (also >of our own) with 64 Mb which is certainly an improvement and a kind of >standard machine today. However it's our goal to play the CCL >competition on the newest and fastest Pc's available. Sponsor requests >for that are already gone out and are taken into consideration by >several leading hardware companies. > > --------------------- Hi Ed, This sounds exciting to me. My main comment is that I don't see Fritz on the list. I have been pretty much skipping over the "Fritz sucks" postings but seem to remember there is a problem with the autotester. Is that why Fritz is not on the list? However without Fritz (whether Fritz is really best or not), the results will be much less meaningful "program X won the CCL league championship BUT..." I hope this doesn't come across as "we had our own championship and didn't invite Fritz." Also you mentioned exactly equal hardware to make things "fair." But I consider this actually unfair. For instance, I cannot see a 64 bit program (like Crafty, Cilkchess or Dark Thought) being able to compete "fairly" being forced to use 32 bit technology which will soon die, especially when the next generation intels arrive. Even if your program runs on your specified hardware, it may not be optimal for certain ones, do you consider this good? In my humble opinion it supports only a certain way of writing a chess program and discourages others. You did not specify the Operating system either, but I will assume this will be another constraint we must conform to. The idea of using a specific protocol (auto-232) is an excellent idea and in my opinion this is something each program really should conform to. This (perhaps overly critical) post is not meant to discourage this competition. I think it's a great idea in fact because it targets the platform and OS most of us use. Since I much prefer Unix/Linux and better hardware than Pentiums, I may not be completely objective about this issue. On the other hand I expect to get a thousand hate mail responses saying how important it is to conform to the microsoft/intel view of the world. There will be no war because I will not respond. I will point out in advance however, that I definitely DO see the value of standards and conformity as well as the limitations. I'm not a "microsoft sucks" guy either, this is an important and very reasonable platform, it's just not the one I choose to embrace nor is it the only one other people use. Why exclude the rest of the world? But I see it would be a logistical nightmare to try and support the rest of the world too. For what it's worth, my idea of a fair competition (to replace SSDF?) is the following general plan: . Testing to be performed by objective party, chess program authors (or their companies) should not be involved in the actual testing, although their input might be sought (open to all.) . Standardize the interface (auto-232 for instance.) . Allow ANY hardware/software whatsoever, if it is available to everyone. This means the program must be either commercial, or downloadable by everyone. . The author must specify the hardware platform to be run on. The author can PROVIDE a platform if the testers do not have it available. The testing organization would provide 2 or 3 "standard" configurations that are popular. . An entrant is a hardware/software combination. The same program running on another platform is a different entity. . Commercial entries clearly specified and purchased off the shelf. Same for public domain versions. No special or secret versions. . Testing methodology should be completely deterministic. There should be no human decision making involved in pairing decisions. . Testing software should forfeit games when time is exceeded, just like in real tournaments. Unfinished games should also be counted as forfeits if the fault is a particular program's. . Programs should not be moved around or reset. If learning is part of the program, it should start with a clear state, and then allowed to continue unimpeded. . All results to be posted (including time forfeits.) I would think it would be in everyones best interest (except the software companies) to include any public domain program like Crafty or Gnuchess if the author(s) submits it and it's available to everyone. For instance Cilkchess would not be testable unless I made it public domain, and I would have to provide hardware if I wanted the parallel version to be tested. - Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.