Author: Ratko V Tomic
Date: 10:07:41 08/09/99
The monthly archive files are stored as zip files
containing hundreds of small txt files. This
fragmentation has a large negative effect on both
compression ratio (thus download time) and the disk
space used by the decompressed txt files.
Additionally searching for some text in the large
number of txt files is slower compared to the same
text in a single file.
SUGGESTION: Merge all txt files from a single
monthly archive into a single (or a few) txt files
and ZIP the merged file.
TEST: As an example of the savings, I took one
typicall monthly archive, M981130.ZIP which had
1147k, containing 1078 small text files.
Archive M981130.ZIP: 1147k B1
Decompressed TXT files: 2104k
Disk space used by TXT: 17248k A1 (waste due to disk granularity)
-----------------------------------
Merged X.TXT file: 2104k
Disk used by X.TXT: 2112k A2
Zipped X.TXT size: 569k B2
-----------------------------------
Ratio A1/A2: 8.2
Ratio B1/B2: 2.0
-----------------------------------
Therefore, the decompresed files would use
8.2 times less disk space than the current
fragmented scheme (and have much faster searches).
The compressed files would save half the space for
the zip files and take half the time to download.
In order to accomodate users who cannot read with
their editors files of one or more Mb, instead of
single monthly TXT file, they can be merged into
chunks of 2-300k per file, which would still
retain all the savings given in the example.
The whole process of conversion can be automated,
the way depends on the server's operating system.
(If the site webmaster uses Windows/MS-DOS system
I could provide utilities for fast automatic
conversion.)
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.