Author: Dann Corbit
Date: 21:11:39 10/27/05
Go up one level in this thread
On October 28, 2005 at 00:10:11, Dann Corbit wrote: >On October 27, 2005 at 23:57:24, Dann Corbit wrote: > >>On October 27, 2005 at 23:45:08, Yakov Konoval wrote: >> >>>On October 27, 2005 at 22:17:18, Dann Corbit wrote: >>> >>>> >>>>What I would want from a set of EGTB files is the fastest possible access. >>>>So if decomression of the block was slower than reading the decompressed block, >>>>then the compression is literally "a waste of time." >>>> >>> >>>The krbnkrb EGTB has size about 160 GB (with using zlib.dll). With using >>>LZMA-algorithm this size can be reduced approximately up to 100 GB. >>>From the other side the fastest possible access is very important for a chess >>>playing-program, but not so important for data-mining programs. >>> >>>>>If you need provably the best compression, then look no further: >>>>>http://www.ii.uni.wroc.pl/~inikep/research/pasqda41b.zip >>>>> >>>>>According to this document: >>>>>http://www.maximumcompression.com/data/doc.php >>>>>pasqda41b achieved 1.3552 bits per byte against a large body of inputs. >>>>> >>>>>Comes with source code. But expect it to be a slow, slow memory hog. >>>> >>>>I looked up that program (pasqda41b), and it is designed for text files. So it >>>>probably won't work for EGTB files. >>> >>>Usually EGTBs can be compressed much better than 8/1.3552 times. >>>I know PAQAR4 archiver - it really gives optimal compression, but the speed >>>of compression is 4 KB(!!!) / sec. >>>(Probably pasqda41b is written by the authors of paqar4 - I'll check it). >> >>LZMA SDK 4.27 seems an interesting idea to try. >>It is related to the 7-Zip project. >> >>This: >> 144,384 lzma.exe >>compressed into this: >> 61,474 lzma.eze >>42.6% of the former size (too fast to time). >> >>And this: >> 6,810,624 AloneLZMA.ncb >>compressed into this: >> 1,720,055 AloneLZMA.nzb >>25.3% of the former size in 12 seconds for 567,552 bytes/sec > >The test program has a benchmark mode, and it seems to decompress at 20 >MB/Second. > >C:\lzma\SRC\7zip\Compress\LZMA_Alone>lzma b > >LZMA 4.27 Copyright (c) 1999-2005 Igor Pavlov 2005-08-07 > > Compressing Decompressing > > 1035 KB/s 2409 MIPS 19810 KB/s 2236 MIPS > 1034 KB/s 2407 MIPS 19765 KB/s 2231 MIPS > 1035 KB/s 2408 MIPS 19766 KB/s 2231 MIPS > 1018 KB/s 2370 MIPS 18124 KB/s 2045 MIPS > 1003 KB/s 2335 MIPS 19727 KB/s 2226 MIPS > 1001 KB/s 2329 MIPS 19680 KB/s 2221 MIPS > 999 KB/s 2326 MIPS 19474 KB/s 2198 MIPS > 967 KB/s 2250 MIPS 19349 KB/s 2184 MIPS > 1001 KB/s 2329 MIPS 18012 KB/s 2033 MIPS > 1017 KB/s 2368 MIPS 19822 KB/s 2237 MIPS >--------------------------------------------------- > 1011 KB/s 2352 MIPS 19330 KB/s 2181 MIPS Average Here is the source for the SDK: http://prdownloads.sourceforge.net/sevenzip/lzma427.tar.bz2?download
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.