3 ms·
FYI, I forked and improved [1] a Rust implementation that supports both table- and SIMD-accelerated CRC-64/NVME [2] calculations. The SIMD-accelerated (x86/x86_
by onethumb 2y ago
FYI, I forked and improved [1] a Rust implementation that supports both table- and SIMD-accelerated CRC-64/NVME [2] calculations. The SIMD-accelerated (x86/x86_64 and aarch64) version delivers 10X over the table (16-bytes at a time) implementation.
The original implementation [3] did the same thing but for CRC-64/XZ [4].
[1]: https://github.com/awesomized/crc64fast-nvme https://github.com/awesomized/crc64fast-nvme
[2]: https://reveng.sourceforge.io/crc-catalogue/all.htm#crc.cat.crc-64-nvme https://reveng.sourceforge.io/crc-catalogue/all.htm#crc.cat....
[3]: https://github.com/tikv/crc64fast https://github.com/tikv/crc64fast
[4]: https://reveng.sourceforge.io/crc-catalogue/all.htm#crc.cat.crc-64-xz https://reveng.sourceforge.io/crc-catalogue/all.htm#crc.cat....
- Genbox 2y agoHow does it compare to the built-in CRC32 instruction? [1] [1] https://www.intel.com/content/www/us/en/docs/ipp/developer-guide-reference/2021-12/crc32-crc32c.html https://www.intel.com/content/www/us/en/docs/ipp/developer-g...
- onethumb 2y agoThis is computing CRC-64, not CRC-32, so there's not really a comparison. But perhaps most importantly, ours works with a variety of polynomials (there are a lot! [1])... we're just using the NVME one, but it's trivially adaptable to most (all?) of them. (The Intel instruction you link to only works with two - CRC32 and CRC32C) Finally, it's based on Intel's paper [2], so they also believe it's extremely fast. :) [1]: https://reveng.sourceforge.io/crc-catalogue/all.htm https://reveng.sourceforge.io/crc-catalogue/all.htm [2]: https://web.archive.org/web/20131224125630/https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/fast-crc-computation-generic-polynomials-pclmulqdq-paper.pdf https://web.archive.org/web/20131224125630/https://www.intel...