11 ms·
Modern Intel CPUs have an instruction specifically to compute CRC. This instruction is easily consumed through a C++ intrinsic, literally in one line of code. Y
by melted 11y ago
Modern Intel CPUs have an instruction specifically to compute CRC. This instruction is easily consumed through a C++ intrinsic, literally in one line of code. You can't do any better than that, no matter what you use.
- Scaevolus 11y agoThey have lots of data hashed against a different CRC polynomial. The CRC32C acceleration instructions are unsuitable.
- kzrdude 11y agoThey should migrate to crc32c!
- jfb 11y agoIn this particular case, however: [Intel's CRC32 CPU instructions] uses different inputs to the CRC32 algorithm (known as the polynomial) which is apparently more robust, and is used in networks, filesystems, that sort of thing. It gives different results to the "standard" polynomial, typically used in compression. They would have to go back and recompute all their existing stored checksums.
- stelund 11y agoBut they can create new with the new polynomial and try to check against that first. If it fails then fall back to regular polynomial.
- toolslive 11y agoThat's why you have to tag the value with an identifier of the algorithm, so you can change your mind later. The problems they currently face are a direct consequence of an architectural mistake a few years ago.
- jfb 11y agoMaybe; maybe not. Regardless, they aren't in a position to alter their software to the degree required at this time.
- robn_fastmail 11y agoIndeed. As I note in the post, this wasn't something we _needed_ to do at that time, we were just curious. Replacing a function with a faster implementation is trivial; its just another deployment and we do several of those each week. Changing data format adds operational complexity - two codepaths need to be run and maintained and we increase system load and lower response times for the duration. (The data format is versioned, so that's easy - no architectural problem there). Its totally worth doing if you need to, and we're not afraid of that, but you don't do it on a whim - it takes proper planning and testing and needs multiple people involved.