4 ms·

We need a new default hash algorithm for general purpose stuff, because sha512 is a bit too long for copy/paste imho. I propose shac = sha1sum(sha512sum(data))

We need a new default hash algorithm for general purpose stuff, because sha512 is a bit too long for copy/paste imho. I propose shac = sha1sum(sha512sum(data)) - for sha concatenated. Thoughts?

tpetry 6y ago
Sha2 is designed to be truncated to the length you want to have
woodruffw 6y ago
It's probably better to just truncate your SHA-2/512 (or any other SHA-2 variant) output to 160 bits (and add a tag, to avoid confusing people). That being said, most people are probably willing to live with the 8 additional bytes required by SHA-2/224 (which, IIRC, is itself a truncated special case of SHA-2/256).
- grizzles 6y ago
  Good feedback. Thanks tpetry as well. I agree about the tag. It's a cultural oversight that all secure hashes aren't labelled as such. When one is used it ought to be identified by its respective hash algorithm, eg. sha1://7d157d7c000ae27db146575c08ce30df893d3a64
  - Dylan16807 6y ago
    It rarely matters, and that's not an insignificant amount of overhead, especially because you're putting those wasteful slashes in https://www.dailymail.co.uk/sciencetech/article-1220286/Sir-Tim-Berners-Lee-admits-forward-slashes-web-address-mistake.html https://www.dailymail.co.uk/sciencetech/article-1220286/Sir-...
  - sneak 6y ago
    Are you familiar with multihash (which is used extensively in ipfs)? It’s an open standard and library and does exactly this (though the prefix tag itself is not human readable unless you are weird like us). I use it in my projects when I need to serialize hashes, as it has nice go and javascript libraries.
  - jmiserez 6y ago
    There's no need to identify the hash algorithm unless multiple versions are used in the same context, which is rarely the case and often defeats the purpose of using a hash in the first place.
    - michaelt 6y ago
      Loads of things have multiple hash options - from /etc/shadow supporting 6 different password hashes, to Maven storing foo.jar alongside foo.jar.md5 and foo.jar.sha1 and apt-get providing both an MD5sum and a SHA256 in the package index, to TLS and SSH and GPG supporting multiple algorithms for compatibility. None of them do it with "sha1://" though...
geofft 6y ago
If you're picking a new algorithm anyway, go with BLAKE2 or BLAKE3, which are faster than SHA-1 or SHA-2 (and certainly faster than chaining them), intentionally tunable to whatever output size you want, and also flexible / more robust in a few more ways (for instance, it's not susceptible to length extension attacks so you don't need HMAC, the intuitive approach of using hash(key + data) as a MAC is sound). BLAKE2 is surprisingly widely supported - e.g., recent versions of Python have it built into the standard library. BLAKE3 is new as of this year but worth a shot if it's easy for you to incorporate. There's nothing really wrong with SHA-2 for existing applications (well, other than the length extension thing), but if you don't need to be compatible with hashes/applications that already exist, consider BLAKE2 as the current "default" hash.
- IncRnd 6y ago
  Speed of digest creation is not a feature, when considering a security context. If the hash output is not used in a security context then speed is a good thing.
  - cesarb 6y ago
    This is a sadly prevalent misconception. When hashing a password, you want it to be as slow as possible (and salted), so that given the hash, it's harder for an attacker to brute force the password; but when using hashing on other security contexts (for instance, message authentication), you do want it to be as fast as possible. That is: for key derivation from an user-entered password (which might have poor entropy), use algorithms specialized for that like PBKDF2 or Argon2. For everything else, you can use a modern fast cryptographic hash algorithm like BLAKE2 or BLAKE3.
  - geofft 6y ago
    This is incorrect. If you are hashing passwords (or doing equivalent operations - specifically, the inputs are much lower entropy than the bit length of the hash output, and attackers have basically unlimited guesses), then you want a password-hashing algorithm, and not a general-purpose cryptographic hash, and you want it to be much, much slower than any general-purpose cryptographic hash - i.e., each attempt should take a good fraction of a second. Argon2 and scrypt are good password-hashing algorithms. None of MDx nor SHA-x nor BLAKEx are suitable here, at least not directly. (You can use them as a primitive in a password-hashing algorithm - for instance, Argon2 happens to use BLAKE2 internally - but then it doesn't matter if the primitive is "too fast," the password-hashing algorithm will just run it multiple times to slow things down.) For all other purposes, including signing and verifying files (documents, executables, etc.), signing and verifying cryptographic certificates, signing and verifying chats, etc., - i.e., cases where the input is higher entropy than the hash length and you're using the hash to allow you to sign/verify a smaller item than the original message - you want a fast, secure cryptographic hash, because that's how you build a fast, reliable system that people will use and not decide they want to use plaintext instead. Also, as it happens, most cryptographic hashes (not password-hashing algorithms) that are slow are also weaker than the cryptographic hashes that are fast. These are two different use cases that demand two different tools, and it's dangerous to lump them in under the terms "security context," because you'll end up with less secure and less usable systems. Password hashing needs a password hashing algorithm to be even slightly secure, and everything else needs a good hash, not just a slow one.
WatchDog 6y ago
How much of exploitability of SHA1 is due to cryptographic weaknesses, and how much are due to it's digest size? If you were able to generalize the SHA1 algorithm, to 256 bits, would it still be in the realm of exploitability?
- IncRnd 6y ago
  Both. The recent issues with SHA1 are that chosen prefix attacks can cause collisions. That's why it's a broken piece of crypto.
- kroeckx 6y ago
  SHA1 is 160 bits and should have a complexity of 2^80 for collision resistance. The last paper puts a chosen-prefix collision at 2^63.4.
IncRnd 6y ago
That entirely depends on what is meant by "general purpose stuff".
raverbashing 6y ago
Well, let's just do it like git that accepts a short suffix (and maybe prefix, I don't remember) of the hash as a commit indicator

We need a new default hash algorithm for general purpose stuff, because sha512 is a bit too long for copy/paste imho. I propose shac = sha1sum(sha512sum(data)) - for sha concatenated. Thoughts?

tpetry 6y ago

Sha2 is designed to be truncated to the length you want to have

woodruffw 6y ago

It's probably better to just truncate your SHA-2/512 (or any other SHA-2 variant) output to 160 bits (and add a tag, to avoid confusing people). That being said, most people are probably willing to live with the 8 additional bytes required by SHA-2/224 (which, IIRC, is itself a truncated special case of SHA-2/256).

grizzles 6y ago
Good feedback. Thanks tpetry as well. I agree about the tag. It's a cultural oversight that all secure hashes aren't labelled as such. When one is used it ought to be identified by its respective hash algorithm, eg. sha1://7d157d7c000ae27db146575c08ce30df893d3a64
- Dylan16807 6y ago
  It rarely matters, and that's not an insignificant amount of overhead, especially because you're putting those wasteful slashes in https://www.dailymail.co.uk/sciencetech/article-1220286/Sir-Tim-Berners-Lee-admits-forward-slashes-web-address-mistake.html https://www.dailymail.co.uk/sciencetech/article-1220286/Sir-...
- sneak 6y ago
  Are you familiar with multihash (which is used extensively in ipfs)? It’s an open standard and library and does exactly this (though the prefix tag itself is not human readable unless you are weird like us). I use it in my projects when I need to serialize hashes, as it has nice go and javascript libraries.
- jmiserez 6y ago
  There's no need to identify the hash algorithm unless multiple versions are used in the same context, which is rarely the case and often defeats the purpose of using a hash in the first place.
  - michaelt 6y ago
    Loads of things have multiple hash options - from /etc/shadow supporting 6 different password hashes, to Maven storing foo.jar alongside foo.jar.md5 and foo.jar.sha1 and apt-get providing both an MD5sum and a SHA256 in the package index, to TLS and SSH and GPG supporting multiple algorithms for compatibility. None of them do it with "sha1://" though...

geofft 6y ago

If you're picking a new algorithm anyway, go with BLAKE2 or BLAKE3, which are faster than SHA-1 or SHA-2 (and certainly faster than chaining them), intentionally tunable to whatever output size you want, and also flexible / more robust in a few more ways (for instance, it's not susceptible to length extension attacks so you don't need HMAC, the intuitive approach of using hash(key + data) as a MAC is sound). BLAKE2 is surprisingly widely supported - e.g., recent versions of Python have it built into the standard library. BLAKE3 is new as of this year but worth a shot if it's easy for you to incorporate. There's nothing really wrong with SHA-2 for existing applications (well, other than the length extension thing), but if you don't need to be compatible with hashes/applications that already exist, consider BLAKE2 as the current "default" hash.

IncRnd 6y ago
Speed of digest creation is not a feature, when considering a security context. If the hash output is not used in a security context then speed is a good thing.
- cesarb 6y ago
  This is a sadly prevalent misconception. When hashing a password, you want it to be as slow as possible (and salted), so that given the hash, it's harder for an attacker to brute force the password; but when using hashing on other security contexts (for instance, message authentication), you do want it to be as fast as possible. That is: for key derivation from an user-entered password (which might have poor entropy), use algorithms specialized for that like PBKDF2 or Argon2. For everything else, you can use a modern fast cryptographic hash algorithm like BLAKE2 or BLAKE3.
- geofft 6y ago
  This is incorrect. If you are hashing passwords (or doing equivalent operations - specifically, the inputs are much lower entropy than the bit length of the hash output, and attackers have basically unlimited guesses), then you want a password-hashing algorithm, and not a general-purpose cryptographic hash, and you want it to be much, much slower than any general-purpose cryptographic hash - i.e., each attempt should take a good fraction of a second. Argon2 and scrypt are good password-hashing algorithms. None of MDx nor SHA-x nor BLAKEx are suitable here, at least not directly. (You can use them as a primitive in a password-hashing algorithm - for instance, Argon2 happens to use BLAKE2 internally - but then it doesn't matter if the primitive is "too fast," the password-hashing algorithm will just run it multiple times to slow things down.) For all other purposes, including signing and verifying files (documents, executables, etc.), signing and verifying cryptographic certificates, signing and verifying chats, etc., - i.e., cases where the input is higher entropy than the hash length and you're using the hash to allow you to sign/verify a smaller item than the original message - you want a fast, secure cryptographic hash, because that's how you build a fast, reliable system that people will use and not decide they want to use plaintext instead. Also, as it happens, most cryptographic hashes (not password-hashing algorithms) that are slow are also weaker than the cryptographic hashes that are fast. These are two different use cases that demand two different tools, and it's dangerous to lump them in under the terms "security context," because you'll end up with less secure and less usable systems. Password hashing needs a password hashing algorithm to be even slightly secure, and everything else needs a good hash, not just a slow one.

WatchDog 6y ago

How much of exploitability of SHA1 is due to cryptographic weaknesses, and how much are due to it's digest size? If you were able to generalize the SHA1 algorithm, to 256 bits, would it still be in the realm of exploitability?

IncRnd 6y ago
Both. The recent issues with SHA1 are that chosen prefix attacks can cause collisions. That's why it's a broken piece of crypto.
kroeckx 6y ago
SHA1 is 160 bits and should have a complexity of 2^80 for collision resistance. The last paper puts a chosen-prefix collision at 2^63.4.

IncRnd 6y ago

That entirely depends on what is meant by "general purpose stuff".

raverbashing 6y ago

Well, let's just do it like git that accepts a short suffix (and maybe prefix, I don't remember) of the hash as a commit indicator