Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Updating engines during tournaments? (Odyssee Tournament)

Author: Robert Hyatt

Date: 07:53:37 03/06/01

Go up one level in this thread


On March 05, 2001 at 22:48:59, Brian Kostick wrote:

>On March 05, 2001 at 16:08:48, Andreas Schwartmann wrote:
>
>>On March 04, 2001 at 11:51:18, Thorsten Czub wrote:
>>
>>>On March 04, 2001 at 10:36:15, Ulrich Tuerke wrote:
>>>
>>>>Die B.27 ist okay, unterscheidet sich kaum von der Paderborn-Version.
>>>>Gruss, Uli
>>>
>>>
>>>brilliant. i ask because i don't want that anybody feels disadvantaged.
>>>if anybody thinks he has a better version he is allowed to upgrade
>>>between the rounds.
>>
>>
>>And that's completely rubbish. If you update engines between rounds, what use is
>>the outcome of such a tournament? There is no consistency in this tourney! An
>>updated engine is a DIFFERENT engine, so you might as well not call it a tourney
>>but a set of engine matches. Hell, you might even start such a "tourney" with
>>Fritz 1 and end up with Fritz 7 ... and what would this say about Fritz's
>>playing strength? He started weak but ended up the winner nontheless? Har har.
>>In my opinon, the engine version that started the tournament should be the very
>>engine that ends it. No changing of horses in midstream or else the results get
>>worthless!
>>
>>Imagine Linares ... Kasparov gets bored in midtourney and gets exchanges by
>>Kramnik ... Shirov does not play to good, so he sends in his brother (does he
>>have one?) ... but that would not be a tourney anymore. Just like your Odyssee
>>with updated engines is no tourney in my opinion.
>>
>>Just my $0.02.
>>
>>Any comments?
>>
>>Andreas
>>
>>
>>
>>www.andreas-schwartmann.de
>
>
>  I think I understand your point. I also note that some are objecting, finding
>no fault with changes between rounds.
>
>  For an example let's say I am to tout Nimzo. Now I may want several tuned
>books, anti-Fritz, anti-Crafty, anti-Whatever. Also I can tuned Nimzo engines
>parameters, if fact use several versions of Nimzo. (99, 2000, 7.32, 8, ect...)
>Its author may even go so far as to change code and recompile between rounds?
>After all this it's fair play? No one objects? Obviously some DO object. Not to
>how an individual host a tournament, but to how the results maybe presented.
>
>  All these changes designed to selectively defeat opponents and claim the glory
>(customer base, money, rating, pride, ect...)? I personally would find it a
>warped glory if it was not the identical product that I released for public
>consumption.


Then warped glory is what you get.  Because it happens all the time.  When we
won our second World Computer Chess Championship with Cray Blitz, we made a
significant change after round 2 where we lost a game we should not have lost
due to a bad change I made prior to the tournament's start.

_every_ computer chess event I have attended has had programmers making changes
between rounds.  From something as simple as a different book to avoid repeating
an opening you just lost in the last round, to adding some analysis for a book
line your next-round opponent seems to want to play given the chance.  The
changes might be more complex modifications to source code to fix a bug or
cover up a hole you found in the last round.  You might find something in your
time allocation code you didn't like and change it to behave more like you think
it should.

Does that mean all the WCCC/WMCCC/ACM event results are bogus?  I don't
think so.  I have played in many _human_ tournaments myself.  And I often spent
a lot of time modifying my _own_ book to get an advantage against a known
opponent.  Humans do this all the time.  Is _that_ also bogus?



>
> IMO, such between round changes make for warped and useless results except to
>maybe some engine authors or a few select others. Definitely not what many of us
>expect when we look at tournament results and try to form some impression or
>conclusion. Regards, Brian K.

I would say at least it is no worse than playing on with an older version that
has had a known bug fixed in a newer version.  Why should a program have to
suffer on with a known problem after it has been fixed?  Commercial programs
release bug fixes as well.  Should _those_ also not be used?



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.