Author: Don Dailey
Date: 10:15:42 05/20/98
Go up one level in this thread
Hi Ed, Thanks for the very reasonable reply. This seems like a pretty good thing to me. I still have mild objections to the forced conformity of platforms but I understand the logistical problems that would be close to impossible to overcome. This is one of those things that a perfect solution may never be found. Your testing will be good, as you say, to help the Window's and Dos people select a good chess program. - Don On May 19, 1998 at 16:35:54, Ed Schröder wrote: >>Posted by Don Dailey on May 19, 1998 at 14:26:48: > >>Hi Ed, > >Hi Don, > >Hope you don't mind I answer with some text of my home page so now >and then. The complete Q&A can be found on my home page. > > >>This sounds exciting to me. My main comment is that I don't see >>Fritz on the list. I have been pretty much skipping over the >>"Fritz sucks" postings but seem to remember there is a problem >>with the autotester. Is that why Fritz is not on the list? > >Q: Why is Fritz not involved in the CCL competition? > >A: >We don't have the Fritz version that supports auto232. It will be >our pleasure to include Fritz in the CCL competition when we have >an auto232 compatible Fritz. > > >>However without Fritz (whether Fritz is really best or not), the >>results will be much less meaningful "program X won the CCL >>league championship BUT..." I hope this doesn't come across >>as "we had our own championship and didn't invite Fritz." > >I think a participating Fritz will make CCL more attractive but >that's up to Chessbase and not up to me. > > >>Also you mentioned exactly equal hardware to make things "fair." >>But I consider this actually unfair. For instance, I cannot see >>a 64 bit program (like Crafty, Cilkchess or Dark Thought) being able >>to compete "fairly" being forced to use 32 bit technology which >>will soon die, especially when the next generation intels arrive. >>Even if your program runs on your specified hardware, it may not >>be optimal for certain ones, do you consider this good? In my >>humble opinion it supports only a certain way of writing a chess >>program and discourages others. > >Well, I don't have a hardware store :) so I have to make some >choices. So I agree programs like yours are being punished for >not being in the 95% area of the Pentium family. But this is >really a problem which can not be solved, I for instance never >complained that SSDF don't test Rebel on AMD because Rebel does >much better on AMD then on Intel chips. > >Q: What about good hardware? > >A: We strive to play the CCL competition on the best available >hardware currently available. The very first games (50 in total) >were played on 4 identical Pentium PRO 200 Mhz machines (of our >own) with 32 Mb. Now matches are played on 4 identical Pentium-II >266 Mhz machines (also of our own) with 64 Mb which is certainly >an improvement and a kind of standard machine today. However it's >our goal to play the CCL competition on the newest and fastest >Pc's available. Sponsor requests for that are already gone out >and are taken into consideration by several leading hardware >companies. > > >>You did not specify the Operating system either, but I will assume >>this will be another constraint we must conform to. The idea >>of using a specific protocol (auto-232) is an excellent idea and >>in my opinion this is something each program really should conform >>to. > >DOS and Windows for the moment. > > >>This (perhaps overly critical) post is not meant to discourage >>this competition. I think it's a great idea in fact because it >>targets the platform and OS most of us use. Since I much >>prefer Unix/Linux and better hardware than Pentiums, I may not be >>completely objective about this issue. On the other hand I >>expect to get a thousand hate mail responses saying how important >>it is to conform to the microsoft/intel view of the world. There >>will be no war because I will not respond. I will point out >>in advance however, that I definitely DO see the value of standards >>and conformity as well as the limitations. I'm not a "microsoft sucks" >>guy either, this is an important and very reasonable platform, it's >>just not the one I choose to embrace nor is it the only one other >>people use. Why exclude the rest of the world? But I see it would >>be a logistical nightmare to try and support the rest of the world >>too. > >Cilkchess is very welcome to participate and I will give you all the >support you need to run Cilkchess optimized but you need to help me >too and Cilkchess should be general available. > >>For what it's worth, my idea of a fair competition (to replace SSDF?) >>is the following general plan: > >>. Testing to be performed by objective party, chess program authors >> (or their companies) should not be involved in the actual testing, >> although their input might be sought (open to all.) > >Agree, quoting my home page... > >The very first thing that will be improved is to move the CCL list >to an independent company, institution or organization to ensure the >creditability AND a continuation for years of this computer-computer >competition. > >At the moment the CCL competition is running in the office of the >Schroder BV, producer of Rebel. It's our first goal with the highest >priority to move the CCL competition to an independent organization >free from any commercial bindings with any of the involved participants. > >We will take 6 months for this process. If we do not manage in these 6 >months to move the CCL list to an independent organization the CCL list >will stop to exist. > >End of quote. > >- Ed - > > >>. Standardize the interface (auto-232 for instance.) >> >>. Allow ANY hardware/software whatsoever, if it is available to >> everyone. This means the program must be either commercial, or >> downloadable by everyone. > >>. The author must specify the hardware platform to be run on. >> The author can PROVIDE a platform if the testers do not have >> it available. The testing organization would provide 2 or 3 >> "standard" configurations that are popular. > >>. An entrant is a hardware/software combination. The same program >> running on another platform is a different entity. > >>. Commercial entries clearly specified and purchased off the shelf. >> Same for public domain versions. No special or secret versions. > >>. Testing methodology should be completely deterministic. There >> should be no human decision making involved in pairing decisions. > >>. Testing software should forfeit games when time is exceeded, >> just like in real tournaments. Unfinished games should also >> be counted as forfeits if the fault is a particular program's. > >>. Programs should not be moved around or reset. If learning is >> part of the program, it should start with a clear state, and >> then allowed to continue unimpeded. > >>. All results to be posted (including time forfeits.) > >>I would think it would be in everyones best interest (except the >>software companies) to include any public domain program like Crafty >>or Gnuchess if the author(s) submits it and it's available to >>everyone. For instance Cilkchess would not be testable unless I >>made it public domain, and I would have to provide hardware if I >>wanted the parallel version to be tested. > >>- Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.