2 ms·
It seems like a more SIMD-friendly version would separately store a byte mask and the packed byte data. Then you can simultaneously expand 16 int32s using e.g.
by fritzo 2y ago
It seems like a more SIMD-friendly version would separately store a byte mask and the packed byte data. Then you can simultaneously expand 16 int32s using e.g. AVX512's _mm512_maskz_expand_epi8(mask,_mm512_loadu_epi8(data)).
- mananaysiempre 2y agoStream VByte[1,2,3] is not far away from that idea, storing the packed lengths in a separate stream from the data bytes. (There’s also a more conventional interleaved design in varint-G8IU, but Stepanov decided to fuck everyone over and patented that.) [1] https://lemire.me/blog/2017/09/27/stream-vbyte-breaking-new-speed-records-for-integer-compression/ https://lemire.me/blog/2017/09/27/stream-vbyte-breaking-new-... [2] https://arxiv.org/abs/1709.08990 https://arxiv.org/abs/1709.08990 [3] https://github.com/lemire/streamvbyte https://github.com/lemire/streamvbyte