Author: Mark Young
Date: 07:50:23 09/09/98
Go up one level in this thread
On September 09, 1998 at 03:31:47, Dirk Frickenschmidt wrote: >On September 08, 1998 at 17:09:12, Mark Young wrote: > >Hi Mark, > > >>On September 08, 1998 at 16:07:31, David Eppstein wrote: >> >>>While I don't disagree with your judgement in this game, I have some concern >>>with your methodology: >>> >>>On September 08, 1998 at 14:02:44, Dirk Frickenschmidt wrote: >>>>In the game below Rebel9 played it's Slav >>>>defence very well, finally reaching an endgame with one pawn up. I already was >>>>convinced that Junior would suffer its first loss against Rebel. >>>... >>>>I judged the game as win for Junior >>>>after move 65; if anyone disagrees it will probably be easy to demonstrate the >>>>win: black cannot prevent to lose one remaining pawn after the other and then >>>>ist mated with king+rook against king. >>> >>>What is the difference between these two situations? >>>In one, you are convinced that Junior will lose, so you keep playing. >>>In the other, you are convinced that Junior will win, so you end the game. >>> >>>In what circumstances do you declare a game over? >>>Shouldn't you wait until the programs themselves checkmate or resign? >>>Otherwise you leave yourself open to questions of partiality. >> > >you wrote: > >>It’s very hard not to pull for one program or another when testing. There are >>many ways to consciously and unconsciously manipulate results when testing >>programs. >>This may explain why people tend to get the results they want when >>playing computer Vs computer games, and only one person is running the >>tournament and playing the games. As imperfect as SSDF is, it is best to have >>the programs play each other with as little human intervention as possible. And >>play many games against each program not just a few. Still any testing is fun >>and interesting to follow. > >I am sorry I can only accept part of what you are saying. > >a) You are completely right concerning human feelings while testing. >I am obviously fascinated by the new program Junior5, as can easily be seen from >my comments and as I do not try to hide at all. >I'd like to add that I was and am similarly fascinated by other programs like >Rebel and Hiarcs, the latter two being my favourite porgrams over the last >years. I always pronounced and pronounce what I like or dislike in quite an open >way. > >b) In contrast to your statement I see no problem for the objectivity of testing >in that, because I always had a sound and controled testing method: > >- using autoplayer games as randomly as they occur (no influence at all from my >side, no skipping of games etc) and the best possible settings for both >machines, playing on equal hardware > >- using standard test positions I sampled over the years, submitting all >programs to all kinds of testing positions, again without picking out certain >psotions which suit a certain program well. > >So while frankly admitting you could read a fascination for Junior5 from my >comments (which would by the way not prevent me from making critical comments >about the same program where I see reasons for that), I don't see how in my >testing there could be >>many ways to consciously and unconsciously manipulate results when testing >>programs. >as you expressed it. > >I am curious to hear from you in which way you could find my testing methodology >biased by human factors in favour of a certain program? >All programs play under the same, clearly defined circumstances. >So please explain... > I do not see anything wrong with the way you tested. I was just explaining to the other poster that people do have feelings when testing programs, but it is to be expected. But this always seems to raise questions about the tester if he seems to pull for one program or another, no matter how fair the test is run. Because there has always been a pattern of the tester’s favorite program doing well in test they run. I could name some recent and past examples of this, but I won’t. Keep the games coming. I like to see any games from any of the newer programs. >Kind regards >from Dirk
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.