2 ms·
Placing a user-supplied dictionary at the front of the archive is essentially how “normal” compression works. The advantage of a user-supplied dictionary is th
by stouset 5y ago
Placing a user-supplied dictionary at the front of the archive is essentially how “normal” compression works.
The advantage of a user-supplied dictionary is that it can be transmitted once, out of band. With this approach it can also be larger than what would make sense for an individual file and it can take advantage of similarity between files.
But if you just send it along with the compressed data anyway, you’re not buying anything extra.
- d110af5ccf 5y agoYes, I realize that it's manually reinventing the wheel. However the context here was supporting random access (and append, truncate, etc) while also using separate formats for archiving and compression. Such an approach is more work but provides clean separation and thus far more flexibility. Bear in mind that the alternative (in context) is throwing a bunch of independently compressed files at something like tar. In the event that the files are small, that will absolutely destroy the compression ratio. If the files are large enough or you only have a few then you could always omit the custom dictionary.