4 ms·
If you look at the code[1], there is basically a special cased/"unrolled" pack and unpack routine for each "bit size" integer that is not already machine addres
by ANTSANTS 14y ago
If you look at the code[1], there is basically a special cased/"unrolled" pack and unpack routine for each "bit size" integer that is not already machine addressable. Each routine seems (disclaimer: I didn't check all of them) to operate on the lowest common multiple of words between "bit size" and word size.
I believe he wrote the rest of the program as a benchmark for his "chunk oriented" routines using large amounts of data. It doesn't make the pack and unpack routines any less suited to processing small amounts of data.
With a little work, you could use those routines to allow random access to a large packed array with a small buffer -- in the simplest implementation, calculate if the integer you're trying to address is in the buffer, if not, unpack the correct chunk and overwrite the buffer with it.
[1] http://pastebin.com/ugGnk00p http://pastebin.com/ugGnk00p
- barrkel 14y agoYou don't need to use the routines; the routines are trivial to write. I expect he gets most of his performance from unrolling and cache effects. My point is rather that if you are in a position to bulk read / write, there is probably a better option.
- ANTSANTS 14y agoI think the author knows there are better options for compressing large streams, but still chose to use the situation as a simple benchmark to test the performance of his (admittedly trivial) packing and unpacking routines. You're right that the benchmark is biased due to cache effects/etc from processing in a large stream, however. A more honest (and useful) benchmark would be to test the performance of packed and unpacked integers under a random access pattern.