3 ms·
For real-time dictionary compression, with infinite-size dictionary (LZ77, using a hash table for O(1) pattern search), you can try: git clone https://github.c
by faragon 9y ago
For real-time dictionary compression, with infinite-size dictionary (LZ77, using a hash table for O(1) pattern search), you can try:
git clone https://github.com/faragon/libsrt https://github.com/faragon/libsrt
cd libsrt
make
./enc -ezh <hugefile >hugefile.z
./enc -dz <hugefile.z >hugefile.again
For 1MB dictionary (faster, because of higher cache hit ratio, and using a much smaller buffer):
./enc -ez <hugefile >hugefile.z
./enc -dz <hugefile.z >hugefile.again
For the source code, check compression (senc_lz_aux()) and decompression (sdec_lz()) functions, much easier to understand than the Zstandard code (because of having less features):
https://github.com/faragon/libsrt/blob/master/src/saux/senc.c https://github.com/faragon/libsrt/blob/master/src/saux/senc....