Author: martin fierz
Date: 15:54:12 09/12/02
Go up one level in this thread
On September 12, 2002 at 07:11:47, Ed Schröder wrote: >On September 11, 2002 at 21:10:31, martin fierz wrote: > >>On September 11, 2002 at 09:02:32, Ed Schröder wrote: >> >> >>>Well, Thorsten achieved something remarkable tuning the Rebel parameters, both >>>his versions perform significant better than the default setting. So far I have, >>> >>>TC-051 : 51-35 59.2% (+55 elo) >>>TC-056 : 43-34 55.8% (+30 elo) >>> >>>Both matches are still in progress but I think the pattern is set. >>> >>>Well done Thorsten. >>> >>>Ed >> >>hi ed, >> >>thanks for the results :-) >>it's so easy... i asked thorsten how many games he plays, and i got no answers, >>only insults. when all i wanted were numbers like these here, which show that he >>is "on to something". i assume this result is against all kinds of other >>engines, right? > >Hola Martin, > >Thorsten and I since day one differ on the subject "how to estimate the strength >of an engine". In general I believe that producing a high volume of games >against other engines is the best way to do it. However I don't think that such >a system is conclusive. > >Take the SSDF list, it provides a sorted list of engines in a certain >environment: 2 PC's, quality of auto232 software, quality of opening books, >quality of learning systems, limited number of opponents. > >There are other strength indications, for instance manual played tournaments >such as the WC cycle. Other ingredients suddenly become important such as the >quality of book preparation, often engines not listed in the SSDF list play a >decisive role in the top ranking of the tournament, by expanding the number of >opponents the result of manual tournaments often looks different than the >ranking of the SSDF list. When is the last time Fritz or Chess Tiger won the >world champion title? It's not only because of too few games played as often >suggested but also about different circumstances. > >Next point, pitting engines against strong humans. The dominating factor of >succes is of an engine is its playing style and its intelligence to avoid >strategic and/or closed positions. Where in comp-comp the search depth often is >the decisive factor in human-comp specific chess knowledge becomes the decisive >factor, we have seen many cases from the past. > >Add-up the attractiveness of an engine. For many especially here at CCC this >issue has a lower value, numbers seem to rule, attractiveness and playing style >comes second or third. Not everybody agrees on that. > >For me all these items count as a whole. I tend to give all these aspects >priorites. Bottom line: I think everybody does the same only the outcome of the >priorities for everybody is different, hence stormy discussions are our part. > >Shalom. > >Ed > hi ed, thanks for the long answer. i completely agree with you. the issue i have with thorsten's tests is that while i think that what he is trying to do is perfectly ok, but you should not make any conclusions based on 10 games. and because i couldn't believe anybody still does that i asked him first to find out if that was really what he was doing... for which he attacked me for "only playing lots of games", without knowing that i also have test sets against which i compare new versions, and that i look at all decisive games in the matches - i don't rely on numbers only either. i once had two versions, one which was playing hyper-aggressive checkers, the other conservative. the hyper-aggressive one was constantly sacrificing material for position (as a result of a bug in my program actually, it was seeing more positional compensation than i wanted to it to see...), and was really just the more interesting engine. the conservative one was marginally better in the engine matches - i released the aggressive one... the bottom line for me is that of course i also toy with my parameters, and of course i also get excited when a new version plays beautiful games and looks good after 10 games. but i never jump to conclusions before the match is over... aloha martin >>aloha >> martin
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.