Author: Enrique Irazoqui
Date: 04:18:30 03/16/98
Go up one level in this thread
On March 16, 1998 at 05:57:35, Ed Schröder wrote: > >Unrealistic view Dirk :)) > >Remember the programmers are to blame and not the SSDF! > >The programmers have taken advantage of AUTO232 adding special >software to their released programs. Rebel8 (just in time) came >with a memory resident learner only active when running AUTO232. >This to prevent Rebel8 being outplayed on SSDF. > >It works good as long as you don't interrupt a match. If a match is >interrupted the learner results are lost. Hiarcs3 has a similar system. >There was no time (due to release pressure) to change the software and >save the learn results permanently on disk. > >Rebel9 came with a book learner. Results are now saved on disk. Avoid >lost games, repeat won games. In principal programmed for the public >but as a CLEAR second goal not to be slaughtered on SSDF. > >Let's face it, book learning is way of out control in comp-comp games. >Just look at the SSDF Fritz5 - Rebel8 (P90) 31.5 - 8.5 and count the >doubles. > >This weekend I checked these 40 games and found out that the Rebel8 >memory resident learner is bypassed by either temporarily interrupting >the F5-R8 match or not starting R8 with the "A" parameter. > >I understand the SSDF tester, he wants to use his PC's for other >purposes too. However my point is that if the F5(200) - R8(P90) >40 game match is replayed without any interruption the 31.5 - 8.5 >score (by far) will not be reproduced because of the R8 memory >resident learner. > >I have checked these 40 games and found out that in almost all >cases the R8 learner was not active. Which means it's a design flaw of R8's learner, and not a wrong way to test by the SSDF. Games are played. In which order they are played it doesn't matter. The problem is that this partcular result is not realistic. Proposing to the SSDF, or to anyone else, to test the way best suited for each program is not realistic either, simply because it is not feasible. There are many programs and many games. Let's assume all programs have a properly designed learner. It writes to disk, it avoids falling for the same losing line. Program A plays program B, both with good learners. Program A tries to repeat a winning line. Program B will avoid it. This way they mutually tend to minimize the effect of learners. As a consequence, the engines are measured and we have no winning double games. >It's my opinion that the solution is very simple and is fully in >the hands of the SSDF themselves. > >#1. Remove all the doubles. This is not practical. First, we have to define what a double is. It was the subject of a long discussion a long time ago. Second, and for the above reasons, competent learners should take care of losing the same game twice. Therefore, no losing doubles anymore. Third, for statistical accuracy we want very many games played. Testers can not check them all one by one. >#2. Accept only general accepted AUTO232 software also available >for the public to check. I am very much in favor, basically to avoid suspicion. But I don't see how this relates to this issue. >SSDF is in full control. >They set the rules. >They have my trust. > >In the meantime you may give me your advice what to do. > >#1. Spent 3-4-5 months of my time to write the perfect comp-comp >learner? Goal: ELO 2900 on SSDF but in reality 400 points less? Learners should neutralize each other. As a result, this 400 points difference is not real. >#2. Forget about SSDF and fully concentrate on the engine and >useful new features? Once the learner is designed, I guess it's the end of the problem. Am I missing something? >#3. Resign from SSDF, Rebel not on SSDF anymore, this in combination >with point #2? Shooting on your own foot? If we have no SSDF, who is going to tell that program A is any better than mass-market programs selling for 25% the price? Besides, I think it is a mistake to blame testers for what is basically a programmer's problem. >#4. Leave things as they are? Definitely. With better learners. Enrique >At the moment I am in favor of #3. If I was not commercial I would >certainly pick option #3 based on idealistic grounds. But I am not >and here the problems starts... > >Any advice is welcome. > >- Ed - > > > >>Kind regards from Dirk
This page took 0.03 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.