Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Substantive way to judge program strength improvement?

Author: Serge Desmarais

Date: 18:54:47 08/27/98

Go up one level in this thread


On August 25, 1998 at 02:43:59, Shaun Graham wrote:

>  Substantive way to judge program strength improvement other than ELO?  I ask
>this question, because most programs coming out will only be 10-20 elo points
>stronger than previous versions.  Since this is the case we could achieve a
>similar gain in ELO by just creating a more highly optimized opening book.
>Further since most new programs always claim to have an improved opening book,
>how can we know there has been any real improvement?  I thought that perhaps you
>could do the no opening book test, but i'm not certain that doing so shows much,
>because  prog A might play the middle game and endgame great, but sucks in the
>opening.  further it's not realistic in the sense, that all strong
>players(humans) actually do have stored mental opening books.


   That is why there exist test suites, like the Louguet's. But since these
tests are public, it is theoretically to fine tune a program to score well in
these tests while it would be weak in other similar but different posiyion (?).
The Nunn's match is another way, since both programs alternatively play White
and Black, starting from a certain known book position, but without using their
own books...

   As what you said about the human players and their mental opening book, they
aren't stuck to play it all the time. I remember an Anand's comment about one of
his game against Lautier (?), I think. He reported having analyzed the position
of the game very deep at home before the tournament (in preparation against
Lautier?) but while he was playing Lautier, he decided to completely ignore the
rest of his deep analysis while in the middle of it and he played (after mental
calculations) a move he never considered at home. After the game, he was
convinced that the move picked up during the game was better than the one
analyzed for hours at home! Computer cannot do that. Other players are/were
known to often improvise durina a game : Alekhine, Larsen, Liuboyevitch etc.
while others stick all the time to their preparation (Kasparov and Karpov are an
example).

Serge Desmarais



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.