Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: CM8000 opening book creation

Author: John Merlino

Date: 11:41:57 12/06/00

Go up one level in this thread


On December 05, 2000 at 15:26:12, Jason Williamson wrote:

>On December 04, 2000 at 19:17:21, John Merlino wrote:
>
>>On December 04, 2000 at 15:30:34, Jason Williamson wrote:
>>
>>>On December 03, 2000 at 14:55:01, John Merlino wrote:
>>>
>>>>On December 03, 2000 at 11:23:19, David Dahlem wrote:
>>>>
>>>>>
>>>>>On December 02, 2000 at 21:09:58, John Merlino wrote:
>>>>>
>>>>>>On December 02, 2000 at 18:23:02, David Dahlem wrote:
>>>>>>
>>>>>>>I have been trying to create an opening book by importing a pgn file. Every
>>>>>>>attempt has resulted in this message - "import error: an error occurred
>>>>>>>importing the pgn file". Has anyone else tried to import pgn files into an
>>>>>>>opening book in CM8000? Does the pgn file have to be free of commentary and/or
>>>>>>>variations?
>>>>>>>
>>>>>>>Dave
>>>>>>
>>>>>>If you think the PGN file is completely free of errors, then E-mail me the file
>>>>>>and I'll pass it on to the development team. Does the error message give you any
>>>>>>indication of WHERE in the file the error occurred, or WHAT the error is?
>>>>>>
>>>>>>jm
>>>>>
>>>>>I decided to try a little test. I thought maybe the problem was with the size of
>>>>>the pgn file so i created some with only 100 games. These were imported without
>>>>>an error message. Then i tried a pgn file with 500 games and got the error
>>>>>message. So it appears the problem with CM8000 importing pgn files into the
>>>>>opening book is with file size. Unless someone wanted to take a large amount of
>>>>>time splitting a large pgn file into very small ones, the import pgn into CM8000
>>>>>opening book is worthless, unless this is addressed in the patch.
>>>>>
>>>>>Dave
>>>>
>>>>It is possible that the number of games is the problem, but I know for a fact
>>>>that I have used it to import PGN files with more than 3,000 games. However, the
>>>>one factor is HOW MANY DIFFERENT OPENINGS are in the PGN file. The more
>>>>disparate the openings, the bigger the opening book will be, and the opening
>>>>book format can only handle a certain number of moves. Since the default depth
>>>>is either 30 or 40 plies (can't remember), this could cause a problem. In other
>>>>words, you can import more games if the games all use the same family of
>>>>openings.
>>>>
>>>>I do know that the patch is going to address this SOMEWHAT by giving the user
>>>>many importing options, such as the depth of the book, what criteria to set the
>>>>frequency of the moves, etc. Hopefully, this will "solve" the problem.
>>>>
>>>>jm
>>>
>>>Oh, are you telling me that the opening book creator is limited in the number of
>>>lines that it can use?  Thats pretty sad.  I wanted to create a massive opening
>>>book from my large pgn.  Yace uses an opening book that is built from a 600 meg
>>>PGN file.  Crafty uses an opening book built from a 1.2 GIG pgn.  In both those
>>>cases though they use extensive pruning of the moves out, and do not store the
>>>whole game of course.
>>>
>>>JW
>>
>>Actually, the limit is the number of MOVES (or nodes), not LINES, but the result
>>is still the same. I believe the maximum number of moves/nodes is about 64K, or
>>around 65,000. Therefore, assuming the default depth for the opening book
>>importer is 40 plies (although it might be 30), if the games in the PGN file are
>>different enough in their openings, then you will only be able to import around
>>1,500 games.
>>
>>This is a worst case scenario, however, and I have found that I am able to
>>import closer to 3,000 games with only the occasional overflow error message. I
>>admit that this is not like being able to import a HUGE PGN file as you mention
>>in those other programs, but the problem stems from the fact that Chessmaster's
>>opening book format has not changed since Chessmaster 5000.
>>
>>I can tell you, though, that the patch is going to provide the user with some
>>ability to set the parameters in which a PGN file is imported, one of them being
>>depth. So, obviously, with a depth of only 15-20, in the worst case you will be
>>able to import twice as many games (in practice, though, probably at least four
>>times as many).
>>
>>jm
>>
>>
>>jm
>
>Why have this limit at all?  Large opening books do not scare me, heck all my
>chessbase engines have 300+ meg books.   Is this limit imposed because of an
>assumption that users don't want to have disk space consumed by a large book?
>In the day of CM5k, this might have been more true, but hard drive space is
>cheap now, and lots of people have 40+ gigs.  My chess software takes up a mere
>10-15 gigs of hd space :)
>
>JW

This limit basically exists because of legacy code. The source for creating many
of the opening books in the game has been lost (since some of the books are over
5 years old) and so the development team is somewhat reluctant to change the
format. This is likely to occur in the next version of Chessmaster, however.

jm



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.