11 ms·

Hashing is not encryption

153 points by eamann 5y ago

yonixw 5y ago
If we think about it in good faith, there are some cases where hash is used for PROTECTION. Like in a password storing of hash(salt+password). So while technically correct, in the day to day language we do use hash as an encryption alternative sometimes.
- KSPAtlas 5y ago
  An encrypted password is worse than a hashed one, since you only need it to be one way
- mypastself 5y ago
  Yeah, the definitions and analogies in the linked article are correct, but I doubt they’d be entirely clear to the uninitiated since they explain the processes but not applications for all three concepts. We get the “how”s but not all of the “why”s. I suspect it’s partly for the reason you’ve mentioned: the line between uses can sometimes get blurry. It’s a decent article that could do with some additional clarifications.
- feldrim 5y ago
  Calling it as an alternative might be an exaggeration. Both are used in security or as a protection like you mentioned. Yet, encryption is a way of storing data for a future[0] use in a secure manner while using the hash, you destroy the data and keep only the footprint. [0] Future can be nanoseconds later in any data-in-transit unlike data-at-rest, so I picked future to address the ambiguity of time measurement in different contexta.
- less_less 5y ago
  The article isn't just technically correct. Hashes and encryption are both parts of cryptographic systems, but their use cases are different. For example, implementing a password database as encrypt(salt, password) would be no good at all: the salt is also stored in the database, so an attacker who steals that database could easily recover everyone's plaintext passwords. Password databases should use a specialized, intentionally slow hash function, with a salt and preferably also with a secret key. That is, they should use something like argon2(secret, salt, password) or HMAC(secret,argon2(salt,password)) -- the latter so that you can keep the HMAC secret on an HSM. This is a common interview question for a reason. A candidate who thinks hashes and encryption are typically alternatives isn't ready to do secure system design.
  - yonixw 5y ago
    I agree, just pointed out where the confusion might come from..
triska 5y ago
A cryptographic hash function H can be used for encryption though, by mimicking a one-time pad: Pick a random integer k, concatenate it with a secret s, hash the result obtaining H(s•k), XOR this with the plaintext yielding the ciphertext C and send the pair (k,C). Pick another integer if you need to encrypt more data. Decryption proceeds in the same way, mutatis mutandis. This is the basic idea behind one of the best currently used ciphers, ChaCha20: It builds on a hash function. From a regulatory and legal perspective, this means that if you want to ban strong encryption, you must ban cryptographic hash functions.
- nathias 5y ago
  is this unbreakable like OTP?
  - triska 5y ago
    No, because the key stream is not random: It depends functionally on the integer which is sent in plain text, and on the shared secret. For example, if you pick the same integer twice with the same secret then the XOR of the two ciphertexts is the XOR of the two plaintexts, thus losing confidentiality.
    - Karliss 5y ago
      Neither is OTP secure if you reuse the key. One of main properties of OTP is that an encrypted message could be decrypted to any message of same size. In case of block cyphers and message longer than one block only a fraction of all messages can be obtained. So if you had enough compute resource(and it wasn't more than atoms in universe) for brute force or weakness was found in cyphers you could find which of the potential decrypted messages makes sense and is more likely.
      - smitty1e 5y ago
        > Neither is OTP secure if you reuse the key. I thought the OT meant "One Time". Reusing the key would take the key from singular to plural usage, no?
soVeryTired 5y ago
I guess the other difference between encryption and hashing is that a hash function need not be one-to-one. Many inputs can result in the same hash, though hopefully it's hard to find collisions. So a hash function is allowed to destroy information, whereas it's pretty important that an encryption algorithm doesn't!
- esamueljohnson 5y ago
  Well, a hash function cannot be one-to-one because of the pigeonhole principle.
  - tux3 5y ago
    It doesn't have to be, in principle. There are hash functions that take fixed size input, and output no smaller (or arbitrarily long) hashes. Look at the hash construction in stream ciphers, for example. The keystream is very long, but the key is short. Or look at a perfect hash function, as used for hash tables.
  - didericis 5y ago
    Theoretically couldn’t there be a hashing algorithm that’s one to one if it always spits out a hash as long or longer than the input message? I’ve never actually walked through the math behind hashing algorithms, but I’m assuming collisions come from truncation. I’m guessing you’re usually not able to know exactly where two inputs that collide for the first n bits end up diverging, so the only way to ensure most hash functions are one to one is if the outputs have infinite length. But, if you had outputs of infinite length for different inputs, eventually they’d have to diverge. Idk if that’s true of all hashing functions/maybe there’s a way to know after what point outputs for different inputs have to diverge for some.
    - AlexSW 5y ago
      The output length/size of a hash function is fixed, whereas it takes an arbitrary-length/size input.
13415 5y ago
Hashing is also not secure hashing, cryptographic hash functions are a small subset of all hash functions. I wish the author would make that clear. I've seen many abuses of cryptographic hash functions where ordinary (though perhaps specialized) hash functions would be more suitable.
- less_less 5y ago
  Yes, but! Many libraries have migrated to cryptographic hash functions even for non-cryptographic use cases such as hash tables. They choose a random key. This mitigates problems where the algorithm performs much worse on pathologically bad, perhaps adversarially chosen, data sets.
  - marginalia_nu 5y ago
    If you are worried about adversarial data, it's probably better to choose a more suitable data structure instead (or change the way you deal with hash collisions: linear probing is probably not a good idea if the data is sketchy). Hash tables are only performant if the hash function is fast, and cryptographic hash functions are anything but.
    - tyingq 5y ago
      Checksums might be a better example. People often use cryptographic hashes for check summing in non-security related scenarios. And, depends on the implementation, but MD5 or SHA-1 is often the same or better performance than a CRC checksum. That shouldn't be the case, but it often is.
  - TillE 5y ago
    A library should absolutely not be using a cryptographic hash for something like a hash table unless they have very particular requirements. You don't want to force a significant performance penalty on all your users when there are good general-purpose hashes like XXH3 out there.
    - 5y ago
gitgud 5y ago
> "Even if you know the algorithm and any secret keys involved, there is no way to un-hash a string. It’s an entirely destructive operation." Hashing is similar to compressing a massive photo into a small thumbnail, it makes it easier and quicker to browse through photos, but you cannot recover the detailed resolution from the thumbnail.
- ericalexander0 5y ago
  >no way to un-hash Wrong. Possible with a rainbow table.
  - 0xdeadb00f 5y ago
    A rainbow table must have the plaintext/hash tuple inside of it. This does not reverse the hash, but confirms that some plaintext hashes to some output.
  - charcircuit 5y ago
    No, you are misunderstanding. There is no inverse function to a hashing algorithm. There is an infinite number of possible inputs for any given hash.
    - charcircuit 5y ago
      algorithm -> function
  - tialaramex 5y ago
    The Rainbow Table is merely a further refinement of a neat trick to improve the performance of a trivial time/space trade on an attack. Ultimately what the Rainbow Table is doing is exactly equivalent to remembering all the inputs you tried and what they hashed to, except it uses less memory/disk than the naive approach and a bit more CPU. Knowing this you can see that it is only practical as an attack if the set of inputs you want to try is so small that you can realistically try all of them and keep the results somewhere. Rainbow Tables got famous because Microsoft's incredibly bad LANMAN hashing scheme only has small inputs (7 bytes, the algorithm runs twice on passwords up to 14 bytes), so you actually can try literally all of them, but at the time a terabyte hard disk was very expensive, Rainbow Tables meant you needed much less disk space to store the resulting data and attack this lousy scheme (but somewhat more CPU to calculate the table).
TorKlingberg 5y ago
It's worth noting that the terminology has changed a bit over time. I.e. the old Unix C function 'crypt' actually does hashing.
- ttyprintk 5y ago
  No, it uses an encryption scheme on the password to derive the field in /etc/passwd. It’s one-way (decrypt is not implemented) but it is encryption based on a rotor-design US Army encryption machine. Edit: Navy, not Army
pgCKIN 5y ago
Funnily enough, hambuger in french is "steak haché" (hashed steak).
- denton-scratch 5y ago
  Not exactly; steak haché is minced (or finely-chopped) steak, as in Steak Tartare for example. You might use steak haché to make a hamburger.
  - OJFord 5y ago
    On a menu, it's a burger. Most likely not in a bun/McDonalds-style though (l'hamburger would be).
    - denton-scratch 5y ago
      OK, I stand corrected.
makach 5y ago
I have to admit that I was incredibly provoked by the title, then I read the article and thought "this is fine". #clickbaited
TacticalCoder 5y ago
It would be nice if TFA added at least once sentence about symmetric encryption vs asymmetric encryption.
mirekrusin 5y ago
I read about hashing, encryption and encoding and somehow at the end I want to be more vegetarian.
- thejackgoode 5y ago
  Makes sense. Once you’re the vegetarianest, you have nothing to hide anymore
ravenstine 5y ago
Referring to hashing as encryption is like calling a fingerprint a lock.
throwaway984393 5y ago
Hashing is basically very, very lossy compression, while encryption is more like a jigsaw puzzle with a billion billion pieces and you hide the map of the original picture.
relaunched 5y ago
You'd be surprised how many people could benefit from this type of information. The only thing is add is that the definition of encryption is incomplete. It seems to focus on symmetric encryption. Asymmetric encryption doesn't require the initial encryption key to get back to the original message. Rather, it uses a key pair - one public and one private.
bgro 5y ago
I don't really understand the confusion around these terms in terms of day-to-day actual work. Is this really a problem? This seems like a trivial question to me so much so that I would (and have) screwed up this question in an interview which I'll discuss here. In my pedantic technical opinion (technical as in literal, not technical-interview), these are all subsets of encryption. Encryption to me is anything that scrambles the data to non-literal-plain-text in a way where you need a key to read it. These are just encryption, but the password is always just the word "password", or for a specific example, the source text. In my continued opinion, can't hashes be "found out" in theory if you had unlimited computing time + unlimited attempts at brute force hashing every string? Encoding is just encrypting the text into a non-literal-plain-text format by using (an extremely weak, known password) to translate into another (computer-readable) language. I don't really have anything to add from the source to this one. Why is my distinction about the definition of encryption important to me? In my opinion, we shouldn't limit our mind to ONLY knowing encryption as a method containing some math formula someone came up with to scramble you data based on an input password. There are a magnitude of ways to encrypt your actions in a more broad sense. For example, what if you identify yourself by handing in a series of paintings to somebody (an authenticator) who physically determines your entry? He can determine if you pass by having knowledge that the order of the paintings and the artist's initials correspond to their position in the alphabet to decrypt your ID number. (Some other tricks could be used to prevent random turn in or duplicates, such as only using a specific style of art, but I'm skipping that for this example.) Is that not an encryption method that accepts a user input and encrypts it with a black box formula to output some code?
- ignoramous 5y ago
  > Encryption to me is anything that scrambles the data to non-literal-plain-text in a way where you need a key to read it. You can't re-read what's hashed, though.
- 3np 5y ago
  In the context of cryptography, encryption has a specific meaning. Your intuitive non-standard definition may be interesting and useful, but its aking to bringing up perceptive hashing like what Apple's been introducing for CPAM on iCloud. At this point it becomes an overloaded term and the meaning depends on context. Words are more useful and efficient if we have a common understanding to stand on. > I don't really understand the confusion around these terms in terms of day-to-day actual work. Is this really a problem? It absolutely is. I've seen software that, instead of salt-hashing passwords in the DB, will encrypt them with an global RSA public key (not only less secure and way less efficient, you now also have an effective undocumented max-length of passwords). Or, way more common, utilizing base64-encoding as "encryption". If understanding of the differences was more widespread, at least these systems may have been less terrible.

153 points by eamann 5y ago

If we think about it in good faith, there are some cases where hash is used for PROTECTION. Like in a password storing of hash(salt+password). So while technically correct, in the day to day language we do use hash as an encryption alternative sometimes.

KSPAtlas 5y ago
An encrypted password is worse than a hashed one, since you only need it to be one way
mypastself 5y ago
Yeah, the definitions and analogies in the linked article are correct, but I doubt they’d be entirely clear to the uninitiated since they explain the processes but not applications for all three concepts. We get the “how”s but not all of the “why”s. I suspect it’s partly for the reason you’ve mentioned: the line between uses can sometimes get blurry. It’s a decent article that could do with some additional clarifications.
feldrim 5y ago
Calling it as an alternative might be an exaggeration. Both are used in security or as a protection like you mentioned. Yet, encryption is a way of storing data for a future[0] use in a secure manner while using the hash, you destroy the data and keep only the footprint. [0] Future can be nanoseconds later in any data-in-transit unlike data-at-rest, so I picked future to address the ambiguity of time measurement in different contexta.
less_less 5y ago
The article isn't just technically correct. Hashes and encryption are both parts of cryptographic systems, but their use cases are different. For example, implementing a password database as encrypt(salt, password) would be no good at all: the salt is also stored in the database, so an attacker who steals that database could easily recover everyone's plaintext passwords. Password databases should use a specialized, intentionally slow hash function, with a salt and preferably also with a secret key. That is, they should use something like argon2(secret, salt, password) or HMAC(secret,argon2(salt,password)) -- the latter so that you can keep the HMAC secret on an HSM. This is a common interview question for a reason. A candidate who thinks hashes and encryption are typically alternatives isn't ready to do secure system design.
- yonixw 5y ago
  I agree, just pointed out where the confusion might come from..

triska 5y ago

A cryptographic hash function H can be used for encryption though, by mimicking a one-time pad: Pick a random integer k, concatenate it with a secret s, hash the result obtaining H(s•k), XOR this with the plaintext yielding the ciphertext C and send the pair (k,C). Pick another integer if you need to encrypt more data. Decryption proceeds in the same way, mutatis mutandis. This is the basic idea behind one of the best currently used ciphers, ChaCha20: It builds on a hash function. From a regulatory and legal perspective, this means that if you want to ban strong encryption, you must ban cryptographic hash functions.

nathias 5y ago
is this unbreakable like OTP?
- triska 5y ago
  No, because the key stream is not random: It depends functionally on the integer which is sent in plain text, and on the shared secret. For example, if you pick the same integer twice with the same secret then the XOR of the two ciphertexts is the XOR of the two plaintexts, thus losing confidentiality.
  - Karliss 5y ago
    Neither is OTP secure if you reuse the key. One of main properties of OTP is that an encrypted message could be decrypted to any message of same size. In case of block cyphers and message longer than one block only a fraction of all messages can be obtained. So if you had enough compute resource(and it wasn't more than atoms in universe) for brute force or weakness was found in cyphers you could find which of the potential decrypted messages makes sense and is more likely.
    - smitty1e 5y ago
      > Neither is OTP secure if you reuse the key. I thought the OT meant "One Time". Reusing the key would take the key from singular to plural usage, no?

soVeryTired 5y ago

I guess the other difference between encryption and hashing is that a hash function need not be one-to-one. Many inputs can result in the same hash, though hopefully it's hard to find collisions. So a hash function is allowed to destroy information, whereas it's pretty important that an encryption algorithm doesn't!

esamueljohnson 5y ago
Well, a hash function cannot be one-to-one because of the pigeonhole principle.
- tux3 5y ago
  It doesn't have to be, in principle. There are hash functions that take fixed size input, and output no smaller (or arbitrarily long) hashes. Look at the hash construction in stream ciphers, for example. The keystream is very long, but the key is short. Or look at a perfect hash function, as used for hash tables.
- didericis 5y ago
  Theoretically couldn’t there be a hashing algorithm that’s one to one if it always spits out a hash as long or longer than the input message? I’ve never actually walked through the math behind hashing algorithms, but I’m assuming collisions come from truncation. I’m guessing you’re usually not able to know exactly where two inputs that collide for the first n bits end up diverging, so the only way to ensure most hash functions are one to one is if the outputs have infinite length. But, if you had outputs of infinite length for different inputs, eventually they’d have to diverge. Idk if that’s true of all hashing functions/maybe there’s a way to know after what point outputs for different inputs have to diverge for some.
  - AlexSW 5y ago
    The output length/size of a hash function is fixed, whereas it takes an arbitrary-length/size input.

13415 5y ago

Hashing is also not secure hashing, cryptographic hash functions are a small subset of all hash functions. I wish the author would make that clear. I've seen many abuses of cryptographic hash functions where ordinary (though perhaps specialized) hash functions would be more suitable.

less_less 5y ago
Yes, but! Many libraries have migrated to cryptographic hash functions even for non-cryptographic use cases such as hash tables. They choose a random key. This mitigates problems where the algorithm performs much worse on pathologically bad, perhaps adversarially chosen, data sets.
- marginalia_nu 5y ago
  If you are worried about adversarial data, it's probably better to choose a more suitable data structure instead (or change the way you deal with hash collisions: linear probing is probably not a good idea if the data is sketchy). Hash tables are only performant if the hash function is fast, and cryptographic hash functions are anything but.
  - tyingq 5y ago
    Checksums might be a better example. People often use cryptographic hashes for check summing in non-security related scenarios. And, depends on the implementation, but MD5 or SHA-1 is often the same or better performance than a CRC checksum. That shouldn't be the case, but it often is.
- TillE 5y ago
  A library should absolutely not be using a cryptographic hash for something like a hash table unless they have very particular requirements. You don't want to force a significant performance penalty on all your users when there are good general-purpose hashes like XXH3 out there.
  - 5y ago

gitgud 5y ago

> "Even if you know the algorithm and any secret keys involved, there is no way to un-hash a string. It’s an entirely destructive operation." Hashing is similar to compressing a massive photo into a small thumbnail, it makes it easier and quicker to browse through photos, but you cannot recover the detailed resolution from the thumbnail.

ericalexander0 5y ago
>no way to un-hash Wrong. Possible with a rainbow table.
- 0xdeadb00f 5y ago
  A rainbow table must have the plaintext/hash tuple inside of it. This does not reverse the hash, but confirms that some plaintext hashes to some output.
- charcircuit 5y ago
  No, you are misunderstanding. There is no inverse function to a hashing algorithm. There is an infinite number of possible inputs for any given hash.
  - charcircuit 5y ago
    algorithm -> function
- tialaramex 5y ago
  The Rainbow Table is merely a further refinement of a neat trick to improve the performance of a trivial time/space trade on an attack. Ultimately what the Rainbow Table is doing is exactly equivalent to remembering all the inputs you tried and what they hashed to, except it uses less memory/disk than the naive approach and a bit more CPU. Knowing this you can see that it is only practical as an attack if the set of inputs you want to try is so small that you can realistically try all of them and keep the results somewhere. Rainbow Tables got famous because Microsoft's incredibly bad LANMAN hashing scheme only has small inputs (7 bytes, the algorithm runs twice on passwords up to 14 bytes), so you actually can try literally all of them, but at the time a terabyte hard disk was very expensive, Rainbow Tables meant you needed much less disk space to store the resulting data and attack this lousy scheme (but somewhat more CPU to calculate the table).

TorKlingberg 5y ago

It's worth noting that the terminology has changed a bit over time. I.e. the old Unix C function 'crypt' actually does hashing.

ttyprintk 5y ago
No, it uses an encryption scheme on the password to derive the field in /etc/passwd. It’s one-way (decrypt is not implemented) but it is encryption based on a rotor-design US Army encryption machine. Edit: Navy, not Army

pgCKIN 5y ago

Funnily enough, hambuger in french is "steak haché" (hashed steak).

denton-scratch 5y ago
Not exactly; steak haché is minced (or finely-chopped) steak, as in Steak Tartare for example. You might use steak haché to make a hamburger.
- OJFord 5y ago
  On a menu, it's a burger. Most likely not in a bun/McDonalds-style though (l'hamburger would be).
  - denton-scratch 5y ago
    OK, I stand corrected.

makach 5y ago

I have to admit that I was incredibly provoked by the title, then I read the article and thought "this is fine". #clickbaited

TacticalCoder 5y ago

It would be nice if TFA added at least once sentence about symmetric encryption vs asymmetric encryption.

mirekrusin 5y ago

I read about hashing, encryption and encoding and somehow at the end I want to be more vegetarian.

thejackgoode 5y ago
Makes sense. Once you’re the vegetarianest, you have nothing to hide anymore

ravenstine 5y ago

Referring to hashing as encryption is like calling a fingerprint a lock.

throwaway984393 5y ago

Hashing is basically very, very lossy compression, while encryption is more like a jigsaw puzzle with a billion billion pieces and you hide the map of the original picture.

relaunched 5y ago

You'd be surprised how many people could benefit from this type of information. The only thing is add is that the definition of encryption is incomplete. It seems to focus on symmetric encryption. Asymmetric encryption doesn't require the initial encryption key to get back to the original message. Rather, it uses a key pair - one public and one private.

bgro 5y ago

I don't really understand the confusion around these terms in terms of day-to-day actual work. Is this really a problem? This seems like a trivial question to me so much so that I would (and have) screwed up this question in an interview which I'll discuss here. In my pedantic technical opinion (technical as in literal, not technical-interview), these are all subsets of encryption. Encryption to me is anything that scrambles the data to non-literal-plain-text in a way where you need a key to read it. These are just encryption, but the password is always just the word "password", or for a specific example, the source text. In my continued opinion, can't hashes be "found out" in theory if you had unlimited computing time + unlimited attempts at brute force hashing every string? Encoding is just encrypting the text into a non-literal-plain-text format by using (an extremely weak, known password) to translate into another (computer-readable) language. I don't really have anything to add from the source to this one. Why is my distinction about the definition of encryption important to me? In my opinion, we shouldn't limit our mind to ONLY knowing encryption as a method containing some math formula someone came up with to scramble you data based on an input password. There are a magnitude of ways to encrypt your actions in a more broad sense. For example, what if you identify yourself by handing in a series of paintings to somebody (an authenticator) who physically determines your entry? He can determine if you pass by having knowledge that the order of the paintings and the artist's initials correspond to their position in the alphabet to decrypt your ID number. (Some other tricks could be used to prevent random turn in or duplicates, such as only using a specific style of art, but I'm skipping that for this example.) Is that not an encryption method that accepts a user input and encrypts it with a black box formula to output some code?

ignoramous 5y ago
> Encryption to me is anything that scrambles the data to non-literal-plain-text in a way where you need a key to read it. You can't re-read what's hashed, though.
3np 5y ago
In the context of cryptography, encryption has a specific meaning. Your intuitive non-standard definition may be interesting and useful, but its aking to bringing up perceptive hashing like what Apple's been introducing for CPAM on iCloud. At this point it becomes an overloaded term and the meaning depends on context. Words are more useful and efficient if we have a common understanding to stand on. > I don't really understand the confusion around these terms in terms of day-to-day actual work. Is this really a problem? It absolutely is. I've seen software that, instead of salt-hashing passwords in the DB, will encrypt them with an global RSA public key (not only less secure and way less efficient, you now also have an effective undocumented max-length of passwords). Or, way more common, utilizing base64-encoding as "encryption". If understanding of the differences was more widespread, at least these systems may have been less terrible.