3 ms·

> using SHA-1 as a non-secure hash, That I actually partly disagree with. There are much, MUCH better performing (faster, less memory use, etc) hashes than SHA

> using SHA-1 as a non-secure hash, That I actually partly disagree with. There are much, MUCH better performing (faster, less memory use, etc) hashes than SHA1 if you don't need security. If you don't need security, and you don't need performance, and you don't have anything else available in libraries, then SHA1 is fine. But that's a pretty rare situation. SHA1 might not be a sign of insecurity, but it's often a sign of poorly thought out design.

hyperman1 5y ago
I'm not going to contradict you. This was just one random example, based on the git/SHA-1 story. To stay in this example: If someone is claiming SHA-1 is faster for them, I'd expect to document what other algorithms they tested, what the numbers were, why better things were unavailable. A future maintainer can check if the decision still makes sense based on the comment. Hence the huge comment from the post above. The problem is, I have a few examples that make a lot more sense, but they can't be published here. They depend on local corporate circumstances and pretty obscure knowledge, and would be undecipherable for the HN crowd.
adrian_b 5y ago
On modern Intel/AMD and ARM CPUs, which have SHA-1 instructions, SHA-1 is very fast, faster than many apparently simpler hashes. There are no other so fast hashes with an output of at least 128 bits that you can find in available libraries and use immediately in your code. The only alternative that is faster and long enough is to use an 128-bit polynomial hash like Poly1305 or the one used inside AES-GCM. Such polynomial hashes are available in various cryptographic libraries, but they are not packaged in a way that would allow them to be used directly in a hashing application, so you might have to extract the code from the library and add an interface to it, to make it usable. Another alternative that has appeared recently is BLAKE3. BLAKE3 can be much faster than SHA-1, but only when it is computed in parallel with a large number of cores. If you do not want your hashing task to entirely take over your computer, SHA-1 remains faster. There are many applications that need long hashes to make negligible the likelihood of a collision, e.g. for file deduplication. For such applications using SHA-1 is by far the least effort choice and there is no disadvantage in using it.
- oconnor663 5y ago
  > BLAKE3 can be much faster than SHA-1, but only when it is computed in parallel with a large number of cores. SIMD optimizations are arguably more important than multithreading. Take a look at these SUPERCOP benchmarks on a recent Intel CPU: https://bench.cr.yp.to/results-hash.html#amd64-pascalinspiron75062n1 https://bench.cr.yp.to/results-hash.html#amd64-pascalinspiro.... Those are all single-threaded measurements, and the SHA-1 implementation there is hardware-accelerated, but BLAKE3 is still almost 4x faster. That speedup comes from AVX-512 vector operations, which serial hashes like SHA-1 can't take (as much) advantage of. Not every caller will want to use the AVX-512 implementation (see "downclocking"), but even with AVX2 BLAKE3 would still be faster. Multithreading is a nice option to have, and of course it looks great in benchmarks if the input is long enough. But general-purpose library APIs like `blake3::hash()` in Rust don't do multithreading by default. They only use SIMD optimizations by default.

hyperman1 5y ago

I'm not going to contradict you. This was just one random example, based on the git/SHA-1 story. To stay in this example: If someone is claiming SHA-1 is faster for them, I'd expect to document what other algorithms they tested, what the numbers were, why better things were unavailable. A future maintainer can check if the decision still makes sense based on the comment. Hence the huge comment from the post above. The problem is, I have a few examples that make a lot more sense, but they can't be published here. They depend on local corporate circumstances and pretty obscure knowledge, and would be undecipherable for the HN crowd.

adrian_b 5y ago

On modern Intel/AMD and ARM CPUs, which have SHA-1 instructions, SHA-1 is very fast, faster than many apparently simpler hashes. There are no other so fast hashes with an output of at least 128 bits that you can find in available libraries and use immediately in your code. The only alternative that is faster and long enough is to use an 128-bit polynomial hash like Poly1305 or the one used inside AES-GCM. Such polynomial hashes are available in various cryptographic libraries, but they are not packaged in a way that would allow them to be used directly in a hashing application, so you might have to extract the code from the library and add an interface to it, to make it usable. Another alternative that has appeared recently is BLAKE3. BLAKE3 can be much faster than SHA-1, but only when it is computed in parallel with a large number of cores. If you do not want your hashing task to entirely take over your computer, SHA-1 remains faster. There are many applications that need long hashes to make negligible the likelihood of a collision, e.g. for file deduplication. For such applications using SHA-1 is by far the least effort choice and there is no disadvantage in using it.

oconnor663 5y ago
> BLAKE3 can be much faster than SHA-1, but only when it is computed in parallel with a large number of cores. SIMD optimizations are arguably more important than multithreading. Take a look at these SUPERCOP benchmarks on a recent Intel CPU: https://bench.cr.yp.to/results-hash.html#amd64-pascalinspiron75062n1 https://bench.cr.yp.to/results-hash.html#amd64-pascalinspiro.... Those are all single-threaded measurements, and the SHA-1 implementation there is hardware-accelerated, but BLAKE3 is still almost 4x faster. That speedup comes from AVX-512 vector operations, which serial hashes like SHA-1 can't take (as much) advantage of. Not every caller will want to use the AVX-512 implementation (see "downclocking"), but even with AVX2 BLAKE3 would still be faster. Multithreading is a nice option to have, and of course it looks great in benchmarks if the input is long enough. But general-purpose library APIs like `blake3::hash()` in Rust don't do multithreading by default. They only use SIMD optimizations by default.