5 ms·
If this hash was used as checksum, how would it compare to CRC?
by snvzz 2y ago
If this hash was used as checksum, how would it compare to CRC?
- vlovich123 2y agoIf you’re asking in terms of quality of the checksum in terms of catching corruption, CRC32 is bad just like any 32-bit hash would be. It’s too easy to generate collisions - other algorithms might perform better but it’s all going to be a marginal improvement - you want 64-bit or 128-bit depending on the expected size of data you’re hashing. CRC also exhibits pretty bad issues that cause it to fail SMHasher (even CRC64) - bias, collisions, etc. So gxhash and XXH3 should both perform significantly better in terms of more reliably detecting changes to the underlying data. In terms of speed, XXH3 is roughly the speed of CRC32 but can output 64 or 128-bit hashes (CRC64 by comparison ~3x slower). This algorithm is ~10x faster than XXH3 for small data sizes and ~2x faster for larger ones so it would be 2x faster than CRC32 for a much higher quality hash and ~6x faster than CRC64 for a much higher quality hash. TLDR: Don’t use CRC, use at least 64-bit or 128-bit checksum, and use a good modern hash algorithm for checksumming like XXH3. I know XXH3 very specifically targets the checksum use-case. I don’t know about gxhash’s suitability for that, but I suspect that it’s equally as fine - it’s really hard to measure the quality of a 64-bit and 128-bit hash unless they’re so bad they fail obvious tests.