Author: Robert Hyatt
Date: 08:02:44 11/29/00
Go up one level in this thread
On November 29, 2000 at 00:49:47, Ed Schröder wrote: >On November 28, 2000 at 20:23:45, Robert Hyatt wrote: > >>On November 28, 2000 at 16:55:46, Ed Schröder wrote: >> >>>On November 28, 2000 at 13:38:42, Robert Hyatt wrote: >>> >>>>On November 28, 2000 at 13:19:11, Ed Schröder wrote: >>>> >>>>>On November 28, 2000 at 13:00:02, Dann Corbit wrote: >>>>> >>>>>>A number of authors have stated in this forum that they use the WAC and other >>>>>>test suites to tune their programs. This may or may not make them play better, >>>>>>as you know. Consider that the Rebel settings for solving positions and the >>>>>>Rebel settings for playing the strongest chess are different. >>>>>> >>>>>>Therefore, to tune purely to solve test suites is probably not the best way to >>>>>>create a strong playing program (though it does produce decent chess). >>>>> >>>>>Using wellknown test positions to test if your program changes do >>>>>a better job is something else than Bob said: >>>>> >>>>>quote: >>>>> >>>>> I have seen (a) programs tuned to choose the right move to improve >>>>> their test result scores artificially; >>>>> >>>>>end quote >>>>> >>>>>Note the word artificially which implies cheating. >>>>> >>>>>This of course may be the case but then I would like to see it >>>>>supported by examples. >>>>> >>>>>Ed >>>> >>>> >>>>Wasn't it the SSDF that found _one_ case of this? Or was it the CSS readers? >>>>I don't remember specifically. But the test was to set up the same position, >>>>with colors reversed, and the program then failed miserably to find the right >>>>move, yet it did it almost instantly with the original position. >>> >>>It was CSS about Rebel 6.0 and they found the precursor of the EOC >>>approach. But it was presented as cheating. It has been discussed >>>a dozen times by now so you should know better in the between time. >>> >> >> >>I didn't say "cheating" did I? > >That is true. > >You used other words :) > > >>I believe I said "a program was tweaked to >>choose the right moves in a test suite for the wrong reason." Whether it was >>accidental or on purpose, that statement is _still_ true. So there is one >>example. > >It was about 2-3 positions of the BT2630 test suite that contains in >total 30 positions. From there a whole suspicion theory was setup why >only 2-3 positions and not the whole test suite. It sounded so convincing >many people believed it and it seriously damaged the reputation of Rebel6. > >It is garbage like you are doing now with Gandalf. Sorry, but I am not doing _anything_ with Gandalf. I haven't accused him of _anything_. I said that often programs get the right move for the wrong reason. I gave some examples of why I don't trust such results. I included _my_ program in the mix as I have posted here _many_ times that Crafty solves XXX but for the wrong reason... To make my comments into something personally directed at the Gandalf author is going way beyond fact, and far into fiction-land. I simply explained why _I_ want to see the _right reason_. As to whether it was "garbage" or not, I don't know. _I_ didn't "discover" the problem. Others did. I didn't test the hypothesis. Others did. It was well-known. And making the statement "it was garbage" is nonsense. It was definitely true. The reason _why_ is something only you can know. But it _did_ happen, which means it _wasn't_ garbage. > > >>There are others. > > >Who? > > > > >>>>Others have tweaked to do better on the Kopec/Bratko test positions. One >>>>company used to advertise their results on the outside of the box the thing >>>>was sold in. > >Which program and what was wrong? > >Examples and evidence please? Fidelity was one. I don't remember the model. I do remember the box. And I asked them about it at an ACM event. Somewhere in the 80's. > > > >>>You accuse without giving examples. You should be moderated :) >>> >>>Ed >> >> >> >>I did _not_ accuse. > > >Yes you did. > >And I am not going to answer your typical next question, "where >have I said that?". I didn't think you would (could). But for the record, an accusation _must_ be directed at someone for it to be an accusation. Again, visit your Webster's... > > >>IE please give me the name of the person I am supposed to >>be accusing of doing something. > >Shifting already has been started. I asked you to give names. And I refused. Because that _would_ turn into an accusation. I gave one name above. There were _others_. > > >>I made a simple, direct, accurate, >>incontrovertible statement about programs being tuned against specific test >>suites. The best example suite was the Kopec/Bratko positions. But there >>have been others. > >Statements without evidence are gossip and slander. Gossip? Maybe. Slander? Go grab your Webster's again. Slander has to be directed toward a specific person or group. Didn't do that in any way. > >Provide evidence but better make sure to have you facts straight before >posting. As a programmer yourself you know there thousands of possible >logical explanations and you could be wrong in the end damaging a program >for no good reason. > >Ed I generally have my facts straight before posting. At least far more often than not. And notice I did _not_ mention any programs for that very reason. You seem to be the one wanting to bring up a specific name.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.