3 ms·
> there are non-cryptographic hashes and checksums that are extraordinarily fast. Is it still true that CRC32 is only about twice as fast as SHA1? Yeah I know
by VLM 4y ago
> there are non-cryptographic hashes and checksums that are extraordinarily fast.
Is it still true that CRC32 is only about twice as fast as SHA1?
Yeah I know the XX hashes are like 30 times faster than SHA1.
A lot depends on instruction set and processor choice.
Maybe another way to put it is I've always been impressed that on small systems SHA1 is enormously longer but only twice as slow as CRC32.
For a lot of interoperable-maxing non-security non-crypto tasks, CRC32 is not a bad choice, if its good enough for Ethernet, zmodem, and mpeg streams its good enough for my telemetry packets LOL. (IIRC iSCSI uses some variant different formulae)
- adrian_b 4y agoCRC32 is fine for kilobyte-sized data packets. For files, it is useless. Even if that was expected, I have computed CRC32 for all the files on an SSD. Of course, I have found thousands of collisions.
- Dylan16807 4y agoBirthday-style collisions don't matter for integrity checking. 32 bits is too small to do the entire job of duplicate detection, but if it's fast enough then you can add a more thorough second pass and still save time.