Author: Robert Hyatt
Date: 14:41:01 03/16/98
Go up one level in this thread
On March 16, 1998 at 13:48:00, Ed Schröder wrote: > >Why not throw learners out? >They hide the real playing strength of a chess engine. >See my below "double game" definition. > I disagree. I would say they "show" the real strength, because if you don't learn, you get killed. If you do learn, you don't get killed... And even against humans that is critical... >Definition of a double game for COMP_X vs COMP_Y games... >Whenever X or Y leaves the book. >Simple as that. >A second situation is a double game. > >Get rid of double games. What do they contribute to comp-comp games? >Nothing in my opinion. > your definition is flawed in the case of Crafty. Crafty not only learns book stuff, it learns after it is out of book... and even if you follow the same book line it may play differently. Your "learning" view is far too narrow, because there are other ways to learn also. See also "knightcap" on the servers. It is a very sophisticated learning program that is getting better and better as it plays. I'd call a "double" a game that is the same from the first move to the last move. Cutting it off anywhere earlier would dilute the type of learning I (and Ferret and probably others) do in addition to my book learning... > >>Second, and for the above reasons, competent learners should take care >>of losing the same game twice. Therefore, no losing doubles anymore. > >Easy to say, difficult to program. Here I disagree. Mine absolutely won't replay a line it loses with, unless that type of learning is disabled... > >Moreover you indirectly say: "The best learner wins the jackpot".... > >How about the chess engine? > >Isn't that the main goal of SSDF? > the engine has to play an opening. if it doesn't know how to learn from mistakes it will get killed by computers and by humans, after enough games are played... > > >>Third, for statistical accuracy we want very many games played. Testers >>can not check them all one by one. > >A little utility can do the job. Secondly the above described double >game definition is implemented since Rebel7. Easy to program. but it is flawed in concept... > > >>>#2. Accept only general accepted AUTO232 software also available >>>for the public to check. > >>I am very much in favor, basically to avoid suspicion. But I don't see >>how this relates to this issue. > >>>SSDF is in full control. >>>They set the rules. >>>They have my trust. > >>>In the meantime you may give me your advice what to do. > >>>#1. Spent 3-4-5 months of my time to write the perfect comp-comp >>>learner? Goal: ELO 2900 on SSDF but in reality 400 points less? > >>Learners should neutralize each other. As a result, this 400 points >>difference is not real. > >If you can recognize the opponent everything is possible. > >Scenario.... > >Play 200-300 games against a SSDF opponent. You then have a learned >book especially tuned on that opponent. Save the new book. > >Repeat that for every expected SSDF opponent. Save the new books. > >Release the program with these optimized opponents books. Being in >AUTO232 recognize the opponent and load the "prepared" book. > >I am not in the mood to put energy in that. It's also a clear cheat. >However if you manage you can enter SSDF with 2900. > > >>>#2. Forget about SSDF and fully concentrate on the engine and >>>useful new features? > >>Once the learner is designed, I guess it's the end of the problem. Am I >>missing something? > >Yes you miss that learners are in the state of the Boris computer of >the late 70ths. So much to improve. > only in some sense. It is is trivial to not play a losing book alternative the second time... which does solve the SSDF issue entirely... > >>>#3. Resign from SSDF, Rebel not on SSDF anymore, this in combination >>>with point #2? > >>Shooting on your own foot? If we have no SSDF, who is going to tell that >>program A is any better than mass-market programs selling for 25% the >>price? > >I don't know how important SSDF is. Based on previous years I would >say a first place on SSDF gives you 10-15% extra sellings. Not much. >I think I will give it a try for the successor of Rebel9 and see if >I can do without SSDF. More features in Rebel that's for sure. > > >>Besides, I think it is a mistake to blame testers for what is basically >>a programmer's problem. > >Who said SSDF is to blame? > >Blame the programmers for their AUTO232 tricks. >Blame SSDF for doing nothing on the problem. >However in the right order..... > > >>>#4. Leave things as they are? > >>Definitely. With better learners. > >I disagree. >Learners hide the strength of an engine in comp-comp games. >My vote is to get rid of them on SSDF simply by excluding double games. >Then (again) you will see the naked ELO of the tested chess engine. > >So what do you want? > >a) The naked ELO of the tested chess engine on SSDF. > >or > >b) An ELO you should add/subtract 50-200 ELO points because of learning. > >You pick.... > >- Ed - > > >>Enrique
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.