Author: Moritz Berger
Date: 04:32:47 08/29/98
Go up one level in this thread
On August 29, 1998 at 07:08:04, Ed Schröder wrote: >Bob is right here. Testing chess program without Permanent Brain (PB) is a >waste of time if you want an accurate result. The time algorithm's in chess >programs are written in cooperation with the PB. Without the PB a chess >program after some time (say after move 20) will automatically become into >time trouble resulting in bad moves. I am listening closely and still taking bets on the outcome of my 40/120 match Crafty 15.17 on PII-400 vs. Junior 5 on P233MMX ... This time with permanent brain enabled, again starting from the 10 Nunn positions with reversed colors (i.e. 20 games total). This is by the way a nice experiment to verify the effect of PB on/off setting, although I fear that if Crafty wins I will only get the "I told you so" statements from Bob and if it loses he will advise me to spend 2 weeks of my life trying to reproduce the result on ICC (thus winning his claim by default since I will not be able to comply ...). >Playing games without the PB I consider the same as playing games >without an opening book. As the opening book the permanent brain >is an essential part of a chess program. > >- Ed - Playing games from a set of opening positions (without book) is useful as it gives you an essentially non-quantitative means of comparing engine performance in well defined thematical testbeds. I.e. if I want to find out if and how you improved Rebel 10 beyond Rebel 9, I will have them play a number of opponents on the Nunn positions and (this is important!) not just add up results (this would be utterly wrong in my opinion since the initial set of positions might be biased in some ways) but take a closer look at deviations and understanding of different types of games (although the Nunn tests usually requires that the winner takes the initiative and therefore might favour some programs above others, e.g. Rebel 10 above Rebel 9 ;-)). The same kind of comparison as for different releases of one program is of course also possible with different hardware setups, time controls, ... In fact, "your own" Jeroen proposed a test set of 50 initial positions, so the basic idea has at least one very important fan even among opening book authors :-) I'm pondering whether I should set up a "Nunn site" on my homepage. I will probably do so as soon as I have results for at least a couple of program pairings at action chess, then iterating testing with regular Standard time controls. One nice side effect is that by using the Nunn positions, nobody can "forget" to enable books or avoid to give *ALL* games played (wins, losses and draws alike) if somebody claims certain results. Since the test has even been kind-of endorsed by Ossi Weiner himself, it's better to use the "nunnery" than to cook up something new and face the old hostile accusations again. Moritz
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.