Author: Russell Reagan
Date: 02:20:38 02/24/05
Go up one level in this thread
On February 23, 2005 at 15:21:30, Scott Gasch wrote: >I hate to ramble about this topic but we are not doing gymnastics at the >olympics where we need judges. There are pretty clear emperical ways to define >a good program: it should do the job, be fast and be small. Readability, >clarity, reusability, approach... these things are nice when you are writing >code at work or hacking on your chess engine. But the question here was "solve >this problem" not "solve this problem using method x (and by the way make nice >readable, reusable code)". > >If you agree about the goals of a good program then how to measure is simple: > >1. We can verify that it does the job easily enough -- compile it and run it >2. We can emperically measure size by compiling all entrants with the same >compiler and settings and comparing the amount of code generated. Counting >lines of C or characters in a file is ridiculous. >3. Because small is not always fast, the last phase is to time the execution of >each entrant. > >The balance between 2. and 3. is debatable but I don't know why we talk about >reusable code or debate people's algorithm choice. (of course this assumes >people are coding in the same language... I don't know how to pick in a pool of >submissions where some people used C and others used Perl... other than to say I >_truly_ appreciate a clever Perl script (and could not, personally, think of one >to solve the problem) but Perl loses on metrics 2. and 3.) > >Scott Usually I enjoy reading anything you post, but I have to put this one up there with the people who complain about CCT and WCCC not being equal hardware :) If it was a contest to see who could write a correct C program which produces the fewest number of x86 AT&T style assembler lines when compiled with gcc 3.4 with the command line options '-O -S', then you would be correct in saying that a good test to determine the winner exists. Unless the contest is that tightly restricted, I don't think there is any good way to determine the "best" entry in some automated manner. You were only able to produce a "score" for most of the entries posted here because it just happens to be the case that most chess programmers use C or C++. Imagine if the question had been posted to a different message board: general programming, python, perl, lisp, unix scripting, and so on. You would get a C program that compiled to 50 assembler lines, and several scripts written in perl, python, and lisp which are all under 10 lines, and some clever guy who submits a Forth program which he claims is best because he has one of those processors that runs Forth directly, and his program is only 25 machine instructions. Vincent writes a parallel version which he runs on a 1024 CPU machine, and of course he claims that is best. Hsu resurrects Deep Blue, and he also thinks that is the best entry. Yet another guy submits a sed script. Finally, some guy manages to produce the correct output by piping /dev/random into egrep and sort -u. So who is the "empirical winner"?
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.