4 ms·
The OP probably finds it funny because Intel processors since Nehalem have had an instruction for computing CRC32.
by woadwarrior01 8y ago
The OP probably finds it funny because Intel processors since Nehalem have had an instruction for computing CRC32.
- pcwalton 8y agoThough, sadly, the CRC32 instruction computes the wrong polynomial for Zip and most other formats. :(
- Twirrim 8y agoWhere is the disparity between the two coming from?
- nigeltao 8y agoThere are two widely used polynomials, IEEE and Castagnoli. These parameterize the CRC-32 algorithm, and are sometimes known as crc32 and crc32c. The IEEE polynomial is used by Bzip2, Ethernet (IEEE 802.3), Gzip, MPEG-2, PNG, SATA, Zip and other formats. The Castagnoli polynomial is used by Btrfs, Ext4, iSCSI, SCTP and other formats. For better or worse, the hardware instruction computes CRC-32-Castagnoli, which means it's not relevant for e.g. Zip.
- Twirrim 8y agoAhh interesting. Thank you. Is there any particular advantage of one polynomial over the other?
- nigeltao 8y agohttps://en.wikipedia.org/wiki/Cyclic_redundancy_check#Standards_and_common_use https://en.wikipedia.org/wiki/Cyclic_redundancy_check#Standa... suggests that Castagnoli has better "error detection capacity". What that means exactly (in terms of information theory) and whether that difference matters in practice is getting beyond my expertise level. In any case, existing formats like Zip are what they are, and as I mentioned in the blog post comments, it is still possible to SIMD-accelerate the IEEE flavor.
- kevin_thibedeau 8y agoIt still should be vectorizable without the dedicated instruction. That isn't evident here so the comparison is questionable.