Author: José Carlos
Date: 10:42:52 02/14/01
Go up one level in this thread
On February 14, 2001 at 10:38:38, Miguel A. Ballicora wrote: >On February 14, 2001 at 08:40:53, José Carlos wrote: > >>On February 14, 2001 at 07:32:53, Uri Blass wrote: >> >>>On February 14, 2001 at 07:05:48, David Dahlem wrote: >>> >>>> >>>>On February 14, 2001 at 06:18:48, Leen Ammeraal wrote: >>>> >>>>>When playing matches, it is obvious that >>>>>the number of games should not be too low >>>>>and the same applies for the time control >>>>>settings. However, my total time to play >>>>>and watch matches is limited so I am always >>>>>wondering what is best, many quick games >>>>>or few more serious games. For example, >>>>>which of the following alternatives is >>>>>best to estimate the relative strength >>>>>of two chess programs? >>>>> >>>>>12 games with 10 s per move, or >>>>>6 games with 20 s per move, or >>>>>4 games with 30 s per move, or >>>>>3 games with 40 s per move, or >>>>>2 games with 60 s per move. >>>>> >>>>>A related question is this: >>>>>If program A is stronger than program B >>>>>in a serious game (with realistic time >>>>>control settings), how likely is it that A >>>>>will also be stronger than B in a quick game? >>>>> >>>>>Leen Ammeraal >>>> >>>>I feel the best way to estimate the relative strength of two programs in just a >>>>few games is to play a few games without opening books. Then you can safely say >>>>the winner is the better of the two at that specific time control, without >>>>playing a lot of games. >>>> >>>>Dave >>> >>>I disagree. >>> >>>If you play without an opening book you miss the fact that one program has a >>>better opening book. >> >> As well, (without opening book) you miss the fact that a programmer who relies >>on his opening book won't implement specific opening knowledge in the program, >>such as development. It's the same case as programmers relying on tablebases. If >>I'm sure my program has 4men tablebases, why bothering teaching the program to >>mate KQ vs KR, for example? >> The opening book _is part of the program_. This has been discussed here many >>times and most people agree in the fact that removing the opening book is >>something like removing extensions, null-move or other stuff. Nonsense. >> >> José C. > >I do not think it is complete nonsense if the goal is to get information >about the evaluation function to improve it. You might not want to fix the >evaluation function based only on these games, but the information is useful. >For instance, what would the program like without the opening book? _In case you have an opening book_, you want it to do that, so you put all your effort on improving the book, making the program play openings that lead the game towards positions _you know_ (because you wrote the evaluation function) your program will "understand". It's obvious that, if your opponent plays 1.a3... 2.a4... 3. Nh3 you'll be out of book, and you'll have to play correctly those positions, but we want to force the opponent to play bad moves to lead the program out of book. In that case, the program should recognize that advantage and win. But that's very different from playing from the beggining without opening book, IMO. And about what "the program likes to play": it's wrong to let it go to positions it seems to like. The program may like to play N+B vs R+P, because its evaluation says it's better, but if you know a trap in the opening where R+P is better than N+B, it can be messy to include it in the opening book, and you _may preffer_ (not always, of course), to avoid the trap in the book. >Will want to gain space? is it careless about the king? >For a very strong program it could be irrelevant (I do not know) but for a >program in the ~2000 it could answer very interesting questions. The result >might be useless, but the observation of the game could be fruitful. I agree here. If you _know_ the result is useless, you can use that kind of games to draw some conclusions about the evaluation. > After all, >the original position is just another position, but much more complex and >flexible that any other in the middlegame... And I have to disagree again. At least, partially. Suppose a postion with 7 queens for each side. It can be legal, and we could say "it's just another postion". But, who cares if his programs plays correctly such position? You know it's almost impossible you reach such a position in a real game. And so, you don't bother implementing special knowledge for it. The starting position is a similar case. You know that you'll hardly play any position similar to the starting one, _because you've got a book for that_. >In my case this point is moot since my program does not have any >opening book. I have to convince myself that what I am watching is useful. >:-) > >Miguel Of course, if you don't have an opening book, you need such special knowledge, plus some randomness, I guess... It's the same case as if we speak about null-move: I don't have null-move in Averno. I simply don't like it. But it would be absolutly nonsense to test my program against Crafty, deactivating Crafty's null-move! Don't you agree... :) José C.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.