4 ms·
Shoco: Compressor for small text strings, with custom model generation tools
- Someone 3y agoFTA: “What is more: An ASCII string is suitable input for the decompressor (which will return the exact same string, of course)” At first, I thought that was giving up lots of valuable encoding space, as it would mean some ASCII strings can be encoded in multiple ways. However, of the 256^n byte sequences of length n only 128^n are full-ASCII. That means the fraction is only 2^-n, and that goes to zero fairly rapidly as n increases. In combination with the self imposed constraints of being a simple and fast compressor, it probably doesn’t matter. What might matter, though, is that this encoder doesn’t round-trip in a decompress-compress cycle. That might cause issues in some use cases, for example, if you fuzz compressed data in testing and check such a decompress-compress cycle. Also FTA: “If the buffer is not large enough for the output, the return value will be bufsize + 1. You might want to allocate a bigger output buffer” That does surprise me. The convention in this kind of C API is to return a value that tells you how large the output buffer has to be to fit the compressed data. That probably doesn’t matter much, though. The compressor guarantees that you won’t need more than twice the input size, and that will be small if you use the library for its intended purpose.