8 ms·

True, but I believe the point is still valid. Given knowledge of cryptographic hashes, and ignorance of HMAC, it is a perfectly reasonable assumption to use a

True, but I believe the point is still valid. Given knowledge of cryptographic hashes, and ignorance of HMAC, it is a perfectly reasonable assumption to use a hash in this way. I think that's a rather common situation, and outcome. And, blindingly obvious in hindsight (I have definitely heard of HMAC before, at least) but new to me too.

tptacek 17y ago
There is no generalist developer in the world that will come up with HMAC when given the "secure hash algorithm" tool and the knowledge that you can "sign" or "protect" a message concatenating it with a key. HMAC is surprisingly intricate. That there is no simple, intuitive alternative to HMAC is a strong reason not to allow developers to work at the "picking algorithms and constructions" level of cryptography. The rule is, "data in motion should be secured with TLS, data at rest should be secured with PGP". What worried me about posts like this is that they communicate the idea that, if you just know a couple more things (like, "HMAC"), you can get crypto right. No. For instance, this article doesn't explain how to safely compare HMACs. The intuitive way to compare securely is drastically insecure. The correct way to do it is surprisingly intricate. If you're typing the letters H-M-A-C into your code, you're doing it wrong.
- ptoomey3 17y ago
  Not to mention, the article mentions using HMAC for his password databases. I am not sure how this is any better than salting. His argument against salting was that the salt is going to be stored somewhere. Well, the key he is going to use with HMAC is going to be stored somewhere too. In general, hashed password storage is a tough game to win in an absolute sense. So, we are effectively left with slowing the attacker down to the point that it isn't worth their while. I didn't see any mention of using iterative hashing (aka stretching) to increase the time required by an attacker to brute-force passwords. I agree with your "TLS in motion and PGP at rest" quote, though I don't know if we have a similar thing we can point to for password storage. Is there a go-to library we can just say "hey, use this". I know we can use bcrypt, or do iterative hashing, etc. But, I don't know off the top of my head what stock library we should tell folks to use so they minimize the chance of shooting themselves in the foot with password storage. Maybe this is something that should be added to keyczar.
  - tptacek 17y ago
    Salted hashes are not that much better than unsalted hashes, but HMAC itself isn't meaningfully different than just using a salted hash. You have to know the key to complete the hash, just like you have to know the salt in a salted scheme. You know you're getting into trouble with someone's understanding of systems security when you start talking about "secret salts".
    - thaumaturgy 17y ago
      > Salted hashes are not that much better than unsalted hashes... This statement made my eyebrows go up, and I pretty closely follow everything you say here on HN, and a lot of the things you say on Matasano. Are you saying "not much better" in the relative sense, as in, "not much better because if all you're doing is salting, then you're still doing it wrong"? Or more absolutely, as in, "salting doesn't gain you anything"?
      - tptacek 17y ago
        For web app passwords, brute force attacks against salted hash files are going to compromise so many passwords that the net effect to your business is going to be bad. With bcrypted or even PBKDF'd passwords, you'll at least have a plausible response to a compromise, and some certainty that hundreds of your users passwords aren't going to be posted to a web page.
        thaumaturgy 17y ago
        Sorry, I still don't understand. Does this assume that all passwords share the same salt, so, not a nonce? Because if each hash has a nonce, I didn't know there was a fast way to brute force a database of those. (Assuming that the developer isn't using md5, of course, but even then I wouldn't expect it to yield a lot of results very quickly.)
        tptacek 17y ago
        Sigh. 1. Take a hash. 2. Start at "aardvark". 3. SHA1 the salt + aadvark. 4. Compare to the hash. 5. Repeat until "zymoscope". It's so weird to me that rainbow tables have so captured people's imaginations that they can't comprehend other attacks to hashes, despite the fact that the brute-force attack I just described was the only way password files were ever cracked for the entirety of the 1990s. It's plenty fast.
- pvg 17y ago
  The rule is, "data in motion should be secured with TLS, data at rest should be secured with PGP". Isn't this more of a heartfelt wish than a rule, though? It's restrictive enough to be impractical, I would guess even for security researchers. Do you use SSH to access remote machines?
  - tptacek 17y ago
    If your problem doesn't fit into PGP or TLS, refactor your problem. There are high-level crypto libraries (though none that I recommend without hesitation) that provide essentially the same features as PGP. If you're going to make a concession, perhaps that's the one you could consider. However: * I don't recommend doing so, and * It's not that much of a concession, because you still have to wrap your application around the cryptosystem, not the other way around.
- swolchok 17y ago
  I just TAed an introductory security class. The crypto project involved building a secure channel, and messages were authenticated with HMAC. We authenticated E(msg) || HMAC-SHA-256(msg) by computing HMAC-SHA-256(D(E(msg)) and bytewise comparing it against the sent HMAC-SHA-256(msg). This was based on Practical Cryptography and seemed fairly intuitive to me. How is it drastically insecure? (I fully admit that I don't know practical crypto, hence the conformance to Practical Cryptography.)
  - tptacek 17y ago
    Bytewise comparison is timeable. Your comparison needs to touch every byte of both the candidate and the real MAC.
    - deleted 17y ago
      [deleted]
    - swolchok 17y ago
      and that's convertible into a fast attack on the MAC key, even through network delay and jitter? (I wouldn't be surprised, now that you've pointed it out.)
      - tptacek 17y ago
        There's a fantastic paper Nate showed me that I need to dig up that established some bounds for timing attacks using statistical filtes. Long story short: you can time low microseconds granularity over the Internet, and you can time nanoseconds over a LAN. Memcmp is just on the threshold of Internet-timeable. And that's memcmp, which screams. But Internet-timeable is irrelevant, because anyone who wants to time your app is just going to get an account at the same hosting provider as you and wind up a GigE hop or two away. Hosting on Slicehost, EC2, Linode, or GAE? That step took 5 minutes and $20. I'd pay $20 to bust up an app I hadn't even heard of, let alone a popular app.
        ephermata 17y ago
- shrughes 17y ago
  The rule is, "data in motion should be secured with TLS, data at rest should be secured with PGP". And even that attitude can be dangerous. A while back I was getting depressed at the usability of tarsnap so I began to think about how I would design some secure backup system on Windows. "Ok," I thought, "I'll build stuff on top of GPG." The trouble is, any way I imagined doing this, even taking out features like deduplication, I would have leaked the filesizes of every file backed up. If the FBI wanted to scan for anybody who had some popular illegal porn collection or whatnot, they could tell just by looking or a certain distribution of chunk sizes. And anybody listening on the network would be able to get a weak idea of the filesizes just based on timing. I have no idea what Tarsnap does about this; I am guessing the blocks it hashes for deduplication purposes aren't the same as the ones it sends to the server, and I presume it doesn't stream stuff to the server as it scans the filesystem. And there are probably several other things it does that I wouldn't even think of. (But even then if some government found that a protest movement was distributing USB sticks with 3.167 GB of information after it gets zipped for backup, it could just look for people uploading that approximate amount of information to the backup server. It seems like people who have a true reason to be paranoid simply shouldn't do online backups.) I would change the rule to "Data in motion should be secured with TLS, data at rest should be secured with PGP, and don't even think about doing anything weird."
  - cperciva 17y ago
    A while back I was getting depressed at the usability of tarsnap Please send me an email with details about the usability issues you encountered. [...] distribution of chunk sizes [...] I have no idea what Tarsnap does about this; I am guessing the blocks it hashes for deduplication purposes aren't the same as the ones it sends to the server, and I presume it doesn't stream stuff to the server as it scans the filesystem. And there are probably several other things it does that I wouldn't even think of. The blocks Tarsnap uses for deduplication are the same as the blocks Tarsnap uploads; but they are encrypted (to avoid leaking file data), identified by HMAC (to avoid leaking information via hash), and the blocking is key dependent, so an attacker can't say "you have chunks of lengths X, Y, and Z, so you probably have this particular block of data". Also, the chunk size (64 kB on average, pre-compression) is small enough that there would be too much noise to pull off such an attack.
- shin_lao 17y ago
  >The rule is, "data in motion should be secured with TLS, data at rest should be secured with PGP". No. The rule is, "ask people who know about security and cryptography to help you solve your problem". That's what we do, and I can tell you that our guidelines are much more complex than simply using TLS or PGP. Actually I think we don't really have guidelines. It's very easy to use TLS and PGP the wrong way.

tptacek 17y ago

There is no generalist developer in the world that will come up with HMAC when given the "secure hash algorithm" tool and the knowledge that you can "sign" or "protect" a message concatenating it with a key. HMAC is surprisingly intricate. That there is no simple, intuitive alternative to HMAC is a strong reason not to allow developers to work at the "picking algorithms and constructions" level of cryptography. The rule is, "data in motion should be secured with TLS, data at rest should be secured with PGP". What worried me about posts like this is that they communicate the idea that, if you just know a couple more things (like, "HMAC"), you can get crypto right. No. For instance, this article doesn't explain how to safely compare HMACs. The intuitive way to compare securely is drastically insecure. The correct way to do it is surprisingly intricate. If you're typing the letters H-M-A-C into your code, you're doing it wrong.

ptoomey3 17y ago
Not to mention, the article mentions using HMAC for his password databases. I am not sure how this is any better than salting. His argument against salting was that the salt is going to be stored somewhere. Well, the key he is going to use with HMAC is going to be stored somewhere too. In general, hashed password storage is a tough game to win in an absolute sense. So, we are effectively left with slowing the attacker down to the point that it isn't worth their while. I didn't see any mention of using iterative hashing (aka stretching) to increase the time required by an attacker to brute-force passwords. I agree with your "TLS in motion and PGP at rest" quote, though I don't know if we have a similar thing we can point to for password storage. Is there a go-to library we can just say "hey, use this". I know we can use bcrypt, or do iterative hashing, etc. But, I don't know off the top of my head what stock library we should tell folks to use so they minimize the chance of shooting themselves in the foot with password storage. Maybe this is something that should be added to keyczar.
- tptacek 17y ago
  Salted hashes are not that much better than unsalted hashes, but HMAC itself isn't meaningfully different than just using a salted hash. You have to know the key to complete the hash, just like you have to know the salt in a salted scheme. You know you're getting into trouble with someone's understanding of systems security when you start talking about "secret salts".
  - thaumaturgy 17y ago
    > Salted hashes are not that much better than unsalted hashes... This statement made my eyebrows go up, and I pretty closely follow everything you say here on HN, and a lot of the things you say on Matasano. Are you saying "not much better" in the relative sense, as in, "not much better because if all you're doing is salting, then you're still doing it wrong"? Or more absolutely, as in, "salting doesn't gain you anything"?
    - tptacek 17y ago
      For web app passwords, brute force attacks against salted hash files are going to compromise so many passwords that the net effect to your business is going to be bad. With bcrypted or even PBKDF'd passwords, you'll at least have a plausible response to a compromise, and some certainty that hundreds of your users passwords aren't going to be posted to a web page.
      - thaumaturgy 17y ago
        Sorry, I still don't understand. Does this assume that all passwords share the same salt, so, not a nonce? Because if each hash has a nonce, I didn't know there was a fast way to brute force a database of those. (Assuming that the developer isn't using md5, of course, but even then I wouldn't expect it to yield a lot of results very quickly.)
        tptacek 17y ago
        Sigh. 1. Take a hash. 2. Start at "aardvark". 3. SHA1 the salt + aadvark. 4. Compare to the hash. 5. Repeat until "zymoscope". It's so weird to me that rainbow tables have so captured people's imaginations that they can't comprehend other attacks to hashes, despite the fact that the brute-force attack I just described was the only way password files were ever cracked for the entirety of the 1990s. It's plenty fast.
pvg 17y ago
The rule is, "data in motion should be secured with TLS, data at rest should be secured with PGP". Isn't this more of a heartfelt wish than a rule, though? It's restrictive enough to be impractical, I would guess even for security researchers. Do you use SSH to access remote machines?
- tptacek 17y ago
  If your problem doesn't fit into PGP or TLS, refactor your problem. There are high-level crypto libraries (though none that I recommend without hesitation) that provide essentially the same features as PGP. If you're going to make a concession, perhaps that's the one you could consider. However: * I don't recommend doing so, and * It's not that much of a concession, because you still have to wrap your application around the cryptosystem, not the other way around.
swolchok 17y ago
I just TAed an introductory security class. The crypto project involved building a secure channel, and messages were authenticated with HMAC. We authenticated E(msg) || HMAC-SHA-256(msg) by computing HMAC-SHA-256(D(E(msg)) and bytewise comparing it against the sent HMAC-SHA-256(msg). This was based on Practical Cryptography and seemed fairly intuitive to me. How is it drastically insecure? (I fully admit that I don't know practical crypto, hence the conformance to Practical Cryptography.)
- tptacek 17y ago
  Bytewise comparison is timeable. Your comparison needs to touch every byte of both the candidate and the real MAC.
  - deleted 17y ago
    [deleted]
  - swolchok 17y ago
    and that's convertible into a fast attack on the MAC key, even through network delay and jitter? (I wouldn't be surprised, now that you've pointed it out.)
    - tptacek 17y ago
      There's a fantastic paper Nate showed me that I need to dig up that established some bounds for timing attacks using statistical filtes. Long story short: you can time low microseconds granularity over the Internet, and you can time nanoseconds over a LAN. Memcmp is just on the threshold of Internet-timeable. And that's memcmp, which screams. But Internet-timeable is irrelevant, because anyone who wants to time your app is just going to get an account at the same hosting provider as you and wind up a GigE hop or two away. Hosting on Slicehost, EC2, Linode, or GAE? That step took 5 minutes and $20. I'd pay $20 to bust up an app I hadn't even heard of, let alone a popular app.
      - ephermata 17y ago
shrughes 17y ago
The rule is, "data in motion should be secured with TLS, data at rest should be secured with PGP". And even that attitude can be dangerous. A while back I was getting depressed at the usability of tarsnap so I began to think about how I would design some secure backup system on Windows. "Ok," I thought, "I'll build stuff on top of GPG." The trouble is, any way I imagined doing this, even taking out features like deduplication, I would have leaked the filesizes of every file backed up. If the FBI wanted to scan for anybody who had some popular illegal porn collection or whatnot, they could tell just by looking or a certain distribution of chunk sizes. And anybody listening on the network would be able to get a weak idea of the filesizes just based on timing. I have no idea what Tarsnap does about this; I am guessing the blocks it hashes for deduplication purposes aren't the same as the ones it sends to the server, and I presume it doesn't stream stuff to the server as it scans the filesystem. And there are probably several other things it does that I wouldn't even think of. (But even then if some government found that a protest movement was distributing USB sticks with 3.167 GB of information after it gets zipped for backup, it could just look for people uploading that approximate amount of information to the backup server. It seems like people who have a true reason to be paranoid simply shouldn't do online backups.) I would change the rule to "Data in motion should be secured with TLS, data at rest should be secured with PGP, and don't even think about doing anything weird."
- cperciva 17y ago
  A while back I was getting depressed at the usability of tarsnap Please send me an email with details about the usability issues you encountered. [...] distribution of chunk sizes [...] I have no idea what Tarsnap does about this; I am guessing the blocks it hashes for deduplication purposes aren't the same as the ones it sends to the server, and I presume it doesn't stream stuff to the server as it scans the filesystem. And there are probably several other things it does that I wouldn't even think of. The blocks Tarsnap uses for deduplication are the same as the blocks Tarsnap uploads; but they are encrypted (to avoid leaking file data), identified by HMAC (to avoid leaking information via hash), and the blocking is key dependent, so an attacker can't say "you have chunks of lengths X, Y, and Z, so you probably have this particular block of data". Also, the chunk size (64 kB on average, pre-compression) is small enough that there would be too much noise to pull off such an attack.
shin_lao 17y ago
>The rule is, "data in motion should be secured with TLS, data at rest should be secured with PGP". No. The rule is, "ask people who know about security and cryptography to help you solve your problem". That's what we do, and I can tell you that our guidelines are much more complex than simply using TLS or PGP. Actually I think we don't really have guidelines. It's very easy to use TLS and PGP the wrong way.