Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: CM8000 opening book creation

Author: John Merlino

Date: 11:41:51 12/06/00

Go up one level in this thread


On December 05, 2000 at 15:26:12, Jason Williamson wrote:

>On December 04, 2000 at 19:17:21, John Merlino wrote:
>
>>On December 04, 2000 at 15:30:34, Jason Williamson wrote:
>>
>>>On December 03, 2000 at 14:55:01, John Merlino wrote:
>>>
>>>>On December 03, 2000 at 11:23:19, David Dahlem wrote:
>>>>
>>>>>
>>>>>On December 02, 2000 at 21:09:58, John Merlino wrote:
>>>>>
>>>>>>On December 02, 2000 at 18:23:02, David Dahlem wrote:
>>>>>>
>>>>>>>I have been trying to create an opening book by importing a pgn file. Every
>>>>>>>attempt has resulted in this message - "import error: an error occurred
>>>>>>>importing the pgn file". Has anyone else tried to import pgn files into an
>>>>>>>opening book in CM8000? Does the pgn file have to be free of commentary and/or
>>>>>>>variations?
>>>>>>>
>>>>>>>Dave
>>>>>>
>>>>>>If you think the PGN file is completely free of errors, then E-mail me the file
>>>>>>and I'll pass it on to the development team. Does the error message give you any
>>>>>>indication of WHERE in the file the error occurred, or WHAT the error is?
>>>>>>
>>>>>>jm
>>>>>
>>>>>I decided to try a little test. I thought maybe the problem was with the size of
>>>>>the pgn file so i created some with only 100 games. These were imported without
>>>>>an error message. Then i tried a pgn file with 500 games and got the error
>>>>>message. So it appears the problem with CM8000 importing pgn files into the
>>>>>opening book is with file size. Unless someone wanted to take a large amount of
>>>>>time splitting a large pgn file into very small ones, the import pgn into CM8000
>>>>>opening book is worthless, unless this is addressed in the patch.
>>>>>
>>>>>Dave
>>>>
>>>>It is possible that the number of games is the problem, but I know for a fact
>>>>that I have used it to import PGN files with more than 3,000 games. However, the
>>>>one factor is HOW MANY DIFFERENT OPENINGS are in the PGN file. The more
>>>>disparate the openings, the bigger the opening book will be, and the opening
>>>>book format can only handle a certain number of moves. Since the default depth
>>>>is either 30 or 40 plies (can't remember), this could cause a problem. In other
>>>>words, you can import more games if the games all use the same family of
>>>>openings.
>>>>
>>>>I do know that the patch is going to address this SOMEWHAT by giving the user
>>>>many importing options, such as the depth of the book, what criteria to set the
>>>>frequency of the moves, etc. Hopefully, this will "solve" the problem.
>>>>
>>>>jm
>>>
>>>Oh, are you telling me that the opening book creator is limited in the number of
>>>lines that it can use?  Thats pretty sad.  I wanted to create a massive opening
>>>book from my large pgn.  Yace uses an opening book that is built from a 600 meg
>>>PGN file.  Crafty uses an opening book built from a 1.2 GIG pgn.  In both those
>>>cases though they use extensive pruning of the moves out, and do not store the
>>>whole game of course.
>>>
>>>JW
>>
>>Actually, the limit is the number of MOVES (or nodes), not LINES, but the result
>>is still the same. I believe the maximum number of moves/nodes is about 64K, or
>>around 65,000. Therefore, assuming the default depth for the opening book
>>importer is 40 plies (although it might be 30), if the games in the PGN file are
>>different enough in their openings, then you will only be able to import around
>>1,500 games.
>>
>>This is a worst case scenario, however, and I have found that I am able to
>>import closer to 3,000 games with only the occasional overflow error message. I
>>admit that this is not like being able to import a HUGE PGN file as you mention
>>in those other programs, but the problem stems from the fact that Chessmaster's
>>opening book format has not changed since Chessmaster 5000.
>>
>>I can tell you, though, that the patch is going to provide the user with some
>>ability to set the parameters in which a PGN file is imported, one of them being
>>depth. So, obviously, with a depth of only 15-20, in the worst case you will be
>>able to import twice as many games (in practice, though, probably at least four
>>times as many).
>>
>>jm
>>
>>
>>jm
>
>Why have this limit at all?  Large opening books do not scare me, heck all my
>chessbase engines have 300+ meg books.   Is this limit imposed because of an
>assumption that users don't want to have disk space consumed by a large book?
>In the day of CM5k, this might have been more true, but hard drive space is
>cheap now, and lots of people have 40+ gigs.  My chess software takes up a mere
>10-15 gigs of hd space :)
>
>JW

This limit basically exists because of legacy code. The source for creating many
of the opening books in the game has been lost (since some of the books are over
5 years old) and so the development team is somewhat reluctant to change the
format. This is likely to occur in the next version of Chessmaster, however.

jm



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.