Author: Ed Schröder
Date: 00:38:37 03/07/01
Go up one level in this thread
On March 06, 2001 at 16:13:12, Robert Hyatt wrote: >On March 06, 2001 at 11:46:36, Brian Kostick wrote: > >>On March 06, 2001 at 10:53:37, Robert Hyatt wrote: >> >>>On March 05, 2001 at 22:48:59, Brian Kostick wrote: >>> >>>>On March 05, 2001 at 16:08:48, Andreas Schwartmann wrote: >>>> >>>>>On March 04, 2001 at 11:51:18, Thorsten Czub wrote: >>>>> >>>>>>On March 04, 2001 at 10:36:15, Ulrich Tuerke wrote: >>>>>> >>>>>>>Die B.27 ist okay, unterscheidet sich kaum von der Paderborn-Version. >>>>>>>Gruss, Uli >>>>>> >>>>>> >>>>>>brilliant. i ask because i don't want that anybody feels disadvantaged. >>>>>>if anybody thinks he has a better version he is allowed to upgrade >>>>>>between the rounds. >>>>> >>>>> >>>>>And that's completely rubbish. If you update engines between rounds, what use is >>>>>the outcome of such a tournament? There is no consistency in this tourney! An >>>>>updated engine is a DIFFERENT engine, so you might as well not call it a tourney >>>>>but a set of engine matches. Hell, you might even start such a "tourney" with >>>>>Fritz 1 and end up with Fritz 7 ... and what would this say about Fritz's >>>>>playing strength? He started weak but ended up the winner nontheless? Har har. >>>>>In my opinon, the engine version that started the tournament should be the very >>>>>engine that ends it. No changing of horses in midstream or else the results get >>>>>worthless! >>>>> >>>>>Imagine Linares ... Kasparov gets bored in midtourney and gets exchanges by >>>>>Kramnik ... Shirov does not play to good, so he sends in his brother (does he >>>>>have one?) ... but that would not be a tourney anymore. Just like your Odyssee >>>>>with updated engines is no tourney in my opinion. >>>>> >>>>>Just my $0.02. >>>>> >>>>>Any comments? >>>>> >>>>>Andreas >>>>> >>>>> >>>>> >>>>>www.andreas-schwartmann.de >>>> >>>> >>>> I think I understand your point. I also note that some are objecting, finding >>>>no fault with changes between rounds. >>>> >>>> For an example let's say I am to tout Nimzo. Now I may want several tuned >>>>books, anti-Fritz, anti-Crafty, anti-Whatever. Also I can tuned Nimzo engines >>>>parameters, if fact use several versions of Nimzo. (99, 2000, 7.32, 8, ect...) >>>>Its author may even go so far as to change code and recompile between rounds? >>>>After all this it's fair play? No one objects? Obviously some DO object. Not to >>>>how an individual host a tournament, but to how the results maybe presented. >>>> >>>> All these changes designed to selectively defeat opponents and claim the glory >>>>(customer base, money, rating, pride, ect...)? I personally would find it a >>>>warped glory if it was not the identical product that I released for public >>>>consumption. >>> >>> >>>Then warped glory is what you get. Because it happens all the time. When we >>>won our second World Computer Chess Championship with Cray Blitz, we made a >>>significant change after round 2 where we lost a game we should not have lost >>>due to a bad change I made prior to the tournament's start. >>> >>>_every_ computer chess event I have attended has had programmers making changes >>>between rounds. From something as simple as a different book to avoid repeating >>>an opening you just lost in the last round, to adding some analysis for a book >>>line your next-round opponent seems to want to play given the chance. The >>>changes might be more complex modifications to source code to fix a bug or >>>cover up a hole you found in the last round. You might find something in your >>>time allocation code you didn't like and change it to behave more like you think >>>it should. >>> >>>Does that mean all the WCCC/WMCCC/ACM event results are bogus? I don't >>>think so. I have played in many _human_ tournaments myself. And I often spent >>>a lot of time modifying my _own_ book to get an advantage against a known >>>opponent. Humans do this all the time. Is _that_ also bogus? >>> >>> >>> >>>> >>>> IMO, such between round changes make for warped and useless results except to >>>>maybe some engine authors or a few select others. Definitely not what many of us >>>>expect when we look at tournament results and try to form some impression or >>>>conclusion. Regards, Brian K. >>> >>>I would say at least it is no worse than playing on with an older version that >>>has had a known bug fixed in a newer version. Why should a program have to >>>suffer on with a known problem after it has been fixed? Commercial programs >>>release bug fixes as well. Should _those_ also not be used? >> >> It is not for you to say that I have to accept someone elses warped glory. I >>maintain the right to be selective. We've seen one major computer chess game >>accept lowered status due to defective time control. Their representative said >>something similar to: We accept this, it's our fault the time control does not >>work properly. A gracious acceptance of reality. Yet when the patch came out of >>course we look foreward to future tournaments with better standing. > >I didn't say you had to accept anything. By your definition, you will be >forced to ignore the results in each of the WCCC events, each of the WMCCC >events, each of the older ACM events, each of the paderborn events, each of >the many other events where computers played but had their authors present and >able to make adjustments to the book or the program between rounds. Note that >that is _every_ major computer event ever held. And you just invalidated every >one of them. > >But continuing, DB was altered a bit after round 2 of the Kasparov match 2. >Rebel was not the same rebel used in every one of the GM challenge games that >Ed played. The SSDF has allowed bugfixes or versions slightly newer than the >versions available commercially. They even allowed non-commercially-available >versions of commercial programs (the famous hidden autoplayer discussion). > >If you use your reasoning, you have _no_ data whatsoever to go on except for >the 'basement tournaments' where someone buys a copy of several programs and >plays them in a tournament "unchanged" between rounds. > > > > >> >> I will view tounament results that are more representative of the user >>product. i.e. Chessfun's results, SSDF, ect... I maintain: If you go to >>tournament unprepared, you lose. As you are fond of pointing out: We agree to >>disagree. With respect for your chess programming, Brian K. > > >You have already disqualified SSDF because they have done exactly what you don't >want to allow, on a few occasions. I would also hope that you don't think the >"same" Kasparov is playing in every tournament he enters. Kasparov is evolving >just like the chess computers are. To compare his play of today with his play >of 20 years ago would show just how far he has come. To pick _any_ single event >and say "this represents _the_ Kasparov" would be wrong. > >Crafty is Crafty... whether it is version 18.3 or 18.4. Just because version >18.4 fixed a small bug in 18.3, does not mean that if you start with 18.3 that >you should stick with it. Humans self-correct bugs in their chess playing all >the time. At least the ones that are getting better do. I don't see why this >should be an issue at all since every available computer event over the last >30 years has been run in _exactly_ the way you don't like. Personally I have >been guilty of only a few between-the-round changes. Chances are you add more >bugs than you remove with so little time to write and test the changes. But I >have sat around in the tournament hall and talked to programmer after programmer >as he madly made changes to fix problems he saw in the previous round... Personally I can live with both systems. I understand the point of view of those who run a tournament with engines that will not change from the beginning to the end. It is a good system provided that the organizer of the tournament has chosen the right version plus the right settings. The way Thorsten runs his tournament is a good system too because it adds several advantages: - The programmer has influence on the engine that participates, he can tell the organizer to play with the version of his choice, tell him what book to use, tell him the hash table size, settings. - Since it is up to the programmer it lowers complaints afterwards. It is always sad to see an organizer spending weeks / months on a tournament and then being faced with criticism such as a) you did not play with the right settings b) you played with the wrong book etc. etc. Such complaints especially when they are justified degenerates the tournament and the end result something nobody is interested in. - Bugs, in the current Odyssey tournament a specific program kept on crashing. What to do? Give it 15 lost games? The programmer send a patch problem solved. It is as Bob says, all world championships work that way. Program fixes, book fixes. Also the SSDF works that way. Nothing new under the sun except that perhaps some people are not aware of how things work in world championship tournaments (or SSDF) because they have never been involved. New is that in Thorsten's case he tries to simulate the world championship system and we are not used to that as we are used to fixed tournaments only. I like both systems and have no preferences. Ed
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.