8 ms·

Don’t Hash Secrets

110 points by davidblair 17y ago

brettbender 17y ago
I feel like this should be titled "Don't hash secrets, unless you actually know what you are doing."
- gcb 17y ago
  Actually he suggests to use something that he has no clue what it's doing. :)
imurray 17y ago
The broader take home message of the article: "Don't invent security protocols (e.g., involving hashing) as there can be surprising weaknesses. Find a well-designed library routine that already does what you want."
- Locke1689 17y ago
  "Don't do crypto yourself."
  - deleted 17y ago
    [deleted]
- arantius 17y ago
  True, but I believe the point is still valid. Given knowledge of cryptographic hashes, and ignorance of HMAC, it is a perfectly reasonable assumption to use a hash in this way. I think that's a rather common situation, and outcome. And, blindingly obvious in hindsight (I have definitely heard of HMAC before, at least) but new to me too.
  - tptacek 17y ago
    There is no generalist developer in the world that will come up with HMAC when given the "secure hash algorithm" tool and the knowledge that you can "sign" or "protect" a message concatenating it with a key. HMAC is surprisingly intricate. That there is no simple, intuitive alternative to HMAC is a strong reason not to allow developers to work at the "picking algorithms and constructions" level of cryptography. The rule is, "data in motion should be secured with TLS, data at rest should be secured with PGP". What worried me about posts like this is that they communicate the idea that, if you just know a couple more things (like, "HMAC"), you can get crypto right. No. For instance, this article doesn't explain how to safely compare HMACs. The intuitive way to compare securely is drastically insecure. The correct way to do it is surprisingly intricate. If you're typing the letters H-M-A-C into your code, you're doing it wrong.
    - ptoomey3 17y ago
      Not to mention, the article mentions using HMAC for his password databases. I am not sure how this is any better than salting. His argument against salting was that the salt is going to be stored somewhere. Well, the key he is going to use with HMAC is going to be stored somewhere too. In general, hashed password storage is a tough game to win in an absolute sense. So, we are effectively left with slowing the attacker down to the point that it isn't worth their while. I didn't see any mention of using iterative hashing (aka stretching) to increase the time required by an attacker to brute-force passwords. I agree with your "TLS in motion and PGP at rest" quote, though I don't know if we have a similar thing we can point to for password storage. Is there a go-to library we can just say "hey, use this". I know we can use bcrypt, or do iterative hashing, etc. But, I don't know off the top of my head what stock library we should tell folks to use so they minimize the chance of shooting themselves in the foot with password storage. Maybe this is something that should be added to keyczar.
      - tptacek 17y ago
        Salted hashes are not that much better than unsalted hashes, but HMAC itself isn't meaningfully different than just using a salted hash. You have to know the key to complete the hash, just like you have to know the salt in a salted scheme. You know you're getting into trouble with someone's understanding of systems security when you start talking about "secret salts".
jf 17y ago
Another post on this topic, with links to research papers: http://rdist.root.org/2009/10/29/stop-using-unsafe-keyed-hashes-use-hmac/ http://rdist.root.org/2009/10/29/stop-using-unsafe-keyed-has...
eplanit 17y ago
Hard to make it past 3 sentences of the arrogant writing style. Such arrogance usually indicates a lack of expertise. The substance of the article confirms that. Yes, it's like "Don't Hash Unless You Really Understand That I'm the Only One Who Really Understands"
cperciva 17y ago
Quoth the article: That means that if you know SHA1(secret || message), then you can compute SHA1(secret || message || ANYTHING), which is a valid signature for message || ANYTHING. So to break this system, you just need to see one signature from SuperAnnoyingPoke, and then you can impersonate SuperAnnoyingPoke for lots of other messages. This is absolutely incorrect. If you know SHA1(secret || message), you can compute SHA1(secret || message || padding || anything), which is quite a different matter. Most useful values of "anything" don't start with such padding. Should anyone be designing a system which relies on this distinction for its security? Of course not. But things are bad enough already without exaggerating the scope of attacks.
- tptacek 17y ago
  You mean useful values of "anything" not including URLs, HTTP cookies, unicode comma-seperated lists, and unicode key=value ASCII pairs, where a small amount of padding isn't going to break anything.
  - cperciva 17y ago
    I don't know about you, but I generally don't allow NUL bytes in my URLs, HTTP cookies, unicode comma-separated lists, or unicode key=value ASCII pairs.
    - tptacek 17y ago
      That's because you're a C programmer. Tarnsap may not have this problem, but Flickr definitely did. I don't know why you think it's a good idea to talk this problem down; it has already been catastrophically bad for some applications.
      - cperciva 17y ago
        I'm not trying to talk the issue down. I'm just saying that it's bad enough already without exaggerating it further.
    - 17y ago
antirez 17y ago
This is what you really want if you care a lot about avoiding the small domain problem (brute forcing H(myguess) so that I finally find H(myguess) == password_hash): Just use an algorithm that is acceptably slow when used to authenticate users, but unacceptably slow when using for brute forcing. For instance HMAC(1,HMAC(2,HMAC(3,MAC(4,HMAC(..N,HMAC("yourPublicZZaaalt","yourpassword")))))) Use N big enough so that you need a few milliseconds to run this code as optimized C. If you are smart, design it so that CUDA wont help. Now you are done. Brute forcing will take just too much if the password is not too obvious. It will be still possible to test 10000 hashes in a reasonable time maybe, but forget to run John The Ripper against it.
- tptacek 17y ago
  Don't use HMAC, but, sure: iterate SHA1 a couple thousand times. This is the approach taken by PBKDF. A better approach is to use bcrypt or scrypt. It's not a security objective of SHA1 or HMAC-SHA1 to take a long time to run --- in fact, they have the opposite design goal. Bcrypt and scrypt have both been designed to be "cryptographically slow": speeding them up in the general case would (forgive an oversimplification) require a new result in cryptography that would be more significant than "optimizing brute force cracking".
  - antirez 17y ago
    I proposed HMAC just in order to avoid that there is some strange property about H(H(H(H(H(...))) probably not true for SHA1 and many other believed secure hash functions and block cyphers. Btw another slow thing that can be used is Blowfish, as it has a slow key scheduling stage, but don't know if it's slow enough. Another approach can be to take any Feistel Network based block cypher and turn it from M rounds to M*N rounds. There are many ways to turn a secure cryptographic primitive into a monkey asses slow cryptographic primitive that seriously limits the effectiveness of brute force.
    - tptacek 17y ago
      Bcrypt is a pessimized version of Blowfish. If you're working at this level of granularity, it should be for a research paper, not something people actually use.
nokya 17y ago
I disagree on that conception...somehow. Can someone explain? http://securecoding.ch/?p=201001290042267 http://securecoding.ch/?p=201001290042267
- dazmax 17y ago
  The article says not to hash shared secrets, it does not argue against hashing passwords (with salts).
  - nokya 17y ago
    Oh. It seemed to me the use cases pretty talked about password hashing. I guess I should re-read :)
oozcitak 17y ago
Such articles leave me with the feeling that there are probably a couple hundred people on earth who truly understand cryptography. The rest of us either blindly follow them (good thing) or try to understand and apply the algorithms ourselves (very bad thing). I'd love to see some sort of crypto-recipes.org site: with a listing for use-cases, recipes underneath and implementations in different languages. I'm sure crypto experts will look down on this, but I believe it would greatly benefit the community.
codahale 17y ago
Good god, no. Here is much better advice, in much shorter form: If you're storing a password, use bcrypt or I'll bite your thumbs off. If you're making sure something hasn't been modified by someone and you absolutely can't use a signed GPG message, use HMAC-SHA256 and a constant-time comparison algorithm or I'll bite your thumbs off. All the business about hash functions is like advice on how to build your own car brakes using a backyard smelter and a sand cast. You might get something of roughly the right shape and weight but someone's going to get hurt down the road.

Don’t Hash Secrets

110 points by davidblair 17y ago

brettbender 17y ago

I feel like this should be titled "Don't hash secrets, unless you actually know what you are doing."

gcb 17y ago
Actually he suggests to use something that he has no clue what it's doing. :)

imurray 17y ago

The broader take home message of the article: "Don't invent security protocols (e.g., involving hashing) as there can be surprising weaknesses. Find a well-designed library routine that already does what you want."

Locke1689 17y ago
"Don't do crypto yourself."
- deleted 17y ago
  [deleted]
arantius 17y ago
True, but I believe the point is still valid. Given knowledge of cryptographic hashes, and ignorance of HMAC, it is a perfectly reasonable assumption to use a hash in this way. I think that's a rather common situation, and outcome. And, blindingly obvious in hindsight (I have definitely heard of HMAC before, at least) but new to me too.
- tptacek 17y ago
  There is no generalist developer in the world that will come up with HMAC when given the "secure hash algorithm" tool and the knowledge that you can "sign" or "protect" a message concatenating it with a key. HMAC is surprisingly intricate. That there is no simple, intuitive alternative to HMAC is a strong reason not to allow developers to work at the "picking algorithms and constructions" level of cryptography. The rule is, "data in motion should be secured with TLS, data at rest should be secured with PGP". What worried me about posts like this is that they communicate the idea that, if you just know a couple more things (like, "HMAC"), you can get crypto right. No. For instance, this article doesn't explain how to safely compare HMACs. The intuitive way to compare securely is drastically insecure. The correct way to do it is surprisingly intricate. If you're typing the letters H-M-A-C into your code, you're doing it wrong.
  - ptoomey3 17y ago
    Not to mention, the article mentions using HMAC for his password databases. I am not sure how this is any better than salting. His argument against salting was that the salt is going to be stored somewhere. Well, the key he is going to use with HMAC is going to be stored somewhere too. In general, hashed password storage is a tough game to win in an absolute sense. So, we are effectively left with slowing the attacker down to the point that it isn't worth their while. I didn't see any mention of using iterative hashing (aka stretching) to increase the time required by an attacker to brute-force passwords. I agree with your "TLS in motion and PGP at rest" quote, though I don't know if we have a similar thing we can point to for password storage. Is there a go-to library we can just say "hey, use this". I know we can use bcrypt, or do iterative hashing, etc. But, I don't know off the top of my head what stock library we should tell folks to use so they minimize the chance of shooting themselves in the foot with password storage. Maybe this is something that should be added to keyczar.
    - tptacek 17y ago
      Salted hashes are not that much better than unsalted hashes, but HMAC itself isn't meaningfully different than just using a salted hash. You have to know the key to complete the hash, just like you have to know the salt in a salted scheme. You know you're getting into trouble with someone's understanding of systems security when you start talking about "secret salts".

jf 17y ago

Another post on this topic, with links to research papers: http://rdist.root.org/2009/10/29/stop-using-unsafe-keyed-hashes-use-hmac/ http://rdist.root.org/2009/10/29/stop-using-unsafe-keyed-has...

eplanit 17y ago

Hard to make it past 3 sentences of the arrogant writing style. Such arrogance usually indicates a lack of expertise. The substance of the article confirms that. Yes, it's like "Don't Hash Unless You Really Understand That I'm the Only One Who Really Understands"

cperciva 17y ago

Quoth the article: That means that if you know SHA1(secret || message), then you can compute SHA1(secret || message || ANYTHING), which is a valid signature for message || ANYTHING. So to break this system, you just need to see one signature from SuperAnnoyingPoke, and then you can impersonate SuperAnnoyingPoke for lots of other messages. This is absolutely incorrect. If you know SHA1(secret || message), you can compute SHA1(secret || message || padding || anything), which is quite a different matter. Most useful values of "anything" don't start with such padding. Should anyone be designing a system which relies on this distinction for its security? Of course not. But things are bad enough already without exaggerating the scope of attacks.

tptacek 17y ago
You mean useful values of "anything" not including URLs, HTTP cookies, unicode comma-seperated lists, and unicode key=value ASCII pairs, where a small amount of padding isn't going to break anything.
- cperciva 17y ago
  I don't know about you, but I generally don't allow NUL bytes in my URLs, HTTP cookies, unicode comma-separated lists, or unicode key=value ASCII pairs.
  - tptacek 17y ago
    That's because you're a C programmer. Tarnsap may not have this problem, but Flickr definitely did. I don't know why you think it's a good idea to talk this problem down; it has already been catastrophically bad for some applications.
    - cperciva 17y ago
      I'm not trying to talk the issue down. I'm just saying that it's bad enough already without exaggerating it further.
  - 17y ago

antirez 17y ago

This is what you really want if you care a lot about avoiding the small domain problem (brute forcing H(myguess) so that I finally find H(myguess) == password_hash): Just use an algorithm that is acceptably slow when used to authenticate users, but unacceptably slow when using for brute forcing. For instance HMAC(1,HMAC(2,HMAC(3,MAC(4,HMAC(..N,HMAC("yourPublicZZaaalt","yourpassword")))))) Use N big enough so that you need a few milliseconds to run this code as optimized C. If you are smart, design it so that CUDA wont help. Now you are done. Brute forcing will take just too much if the password is not too obvious. It will be still possible to test 10000 hashes in a reasonable time maybe, but forget to run John The Ripper against it.

tptacek 17y ago
Don't use HMAC, but, sure: iterate SHA1 a couple thousand times. This is the approach taken by PBKDF. A better approach is to use bcrypt or scrypt. It's not a security objective of SHA1 or HMAC-SHA1 to take a long time to run --- in fact, they have the opposite design goal. Bcrypt and scrypt have both been designed to be "cryptographically slow": speeding them up in the general case would (forgive an oversimplification) require a new result in cryptography that would be more significant than "optimizing brute force cracking".
- antirez 17y ago
  I proposed HMAC just in order to avoid that there is some strange property about H(H(H(H(H(...))) probably not true for SHA1 and many other believed secure hash functions and block cyphers. Btw another slow thing that can be used is Blowfish, as it has a slow key scheduling stage, but don't know if it's slow enough. Another approach can be to take any Feistel Network based block cypher and turn it from M rounds to M*N rounds. There are many ways to turn a secure cryptographic primitive into a monkey asses slow cryptographic primitive that seriously limits the effectiveness of brute force.
  - tptacek 17y ago
    Bcrypt is a pessimized version of Blowfish. If you're working at this level of granularity, it should be for a research paper, not something people actually use.

nokya 17y ago

I disagree on that conception...somehow. Can someone explain? http://securecoding.ch/?p=201001290042267 http://securecoding.ch/?p=201001290042267

dazmax 17y ago
The article says not to hash shared secrets, it does not argue against hashing passwords (with salts).
- nokya 17y ago
  Oh. It seemed to me the use cases pretty talked about password hashing. I guess I should re-read :)

oozcitak 17y ago

Such articles leave me with the feeling that there are probably a couple hundred people on earth who truly understand cryptography. The rest of us either blindly follow them (good thing) or try to understand and apply the algorithms ourselves (very bad thing). I'd love to see some sort of crypto-recipes.org site: with a listing for use-cases, recipes underneath and implementations in different languages. I'm sure crypto experts will look down on this, but I believe it would greatly benefit the community.

codahale 17y ago

Good god, no. Here is much better advice, in much shorter form: If you're storing a password, use bcrypt or I'll bite your thumbs off. If you're making sure something hasn't been modified by someone and you absolutely can't use a signed GPG message, use HMAC-SHA256 and a constant-time comparison algorithm or I'll bite your thumbs off. All the business about hash functions is like advice on how to build your own car brakes using a backyard smelter and a sand cast. You might get something of roughly the right shape and weight but someone's going to get hurt down the road.