7 ms·

Storing Your Data Securely: A Primer

55 points by ShaneWilton 12y ago

itistoday2 12y ago
> These functions belong to a family of functions known as key derivation functions: they use hashing to produce a digest which is suitable for use as a password hash. KDFs are not for storing password hashes, they are for key derivation. This is a subtle point, the main thing is to use KDFs as KDFs (to derive keys that are then used to encrypt data). Details in the first link here (the second is about common mistakes when using scrypt): - http://blog.ircmaxell.com/2014/03/why-i-dont-recommend-scrypt.html http://blog.ircmaxell.com/2014/03/why-i-dont-recommend-scryp... (bad title, he's referring to pass storage, not KDF usage) - http://vnhacker.blogspot.com/2014/04/fairy-tales-in-password-hashing-with.html http://vnhacker.blogspot.com/2014/04/fairy-tales-in-password... Another consideration is plausible deniability (PD) for situations where you're compelled to disclose your password(s). Our company writes Mac encryption software that specializes in this (and it uses scrypt). Here's a list of tools that offer PD (ours is called Espionage): https://en.wikipedia.org/wiki/Deniable_encryption#Software https://en.wikipedia.org/wiki/Deniable_encryption#Software I compared Espionage's PD to TrueCrypt's on reddit: http://www.reddit.com/r/security/comments/2b5icu/major_advancements_in_deniable_encryption_arrive/cj24a1n http://www.reddit.com/r/security/comments/2b5icu/major_advan...
- ShaneWilton 12y ago
  You're right, there's some ambiguous wording on my part. I'll be sure to make that section more precise right now. Looking back on that section, I should also be talking about work factors and such.
  - itistoday2 12y ago
    My apologies, I wasn't right. Please see this reply chain: https://news.ycombinator.com/item?id=8088208 https://news.ycombinator.com/item?id=8088208
    - ShaneWilton 12y ago
      You're right about parts of the post being unclear. I'll be rewriting that paragraph to be more precise regardless.
- tptacek 12y ago
  The first of these two source suggests that with typical settings, due to a quirk with the way scrypt works, it's less secure than bcrypt. The second criticizes an API design with the scrypt code. Neither makes a case that password-based KDFs are "not for storing password hashes". Using a good password KDF as a password authenticator is a fine decision.
  - itistoday2 12y ago
    > The second criticizes an API design with the scrypt code. Yes, I didn't claim otherwise. > Neither makes a case that password-based KDFs are "not for storing password hashes". The first one did quite explicitly for scrypt: "And that's why I don't recommend it for password storage." > Using a good password KDF as a password authenticator is a fine decision. Apparently not so for scrypt, and scrypt is the best KDF I'm aware of (heard something about yescrypt being better but haven't had a chance to look into it). I don't see why a KDF is needed for password storage. What's wrong with SHA256 + random salt? If for some reason you want a slow hash function, use the salt MOD some constant to do several rounds of SHA256 (though I don't see why that would be necessary).
    - tptacek 12y ago
      The last sentence of this comment is exactly why I called it out. If "for some reason" you want a slow hash function? You must have a slow hash function to store a password authenticator. The salt provides virtually no protection; it blocks a single corner-case exploit.
      - itistoday2 12y ago
        > You must have a slow hash function to store a password authenticator. The salt provides virtually no protection; it blocks a single corner-case exploit. You're referring to simple dictionary attacks on a large database of passwords? Even bcrypt won't help much with a password from the top 10k list. I wouldn't say that stopping rainbow tables is "virtually no protection". Please explain what you're referring to.
ctz 12y ago
Not related to the article, but the source: Having my email address on my blog is a vulnerability. Another vulnerability is named 'Rudimentary Scan', meaning 'we looked, and your site has no client-side vulnerabilities'. These issues mean my blog is 'borderline unsafe'. Wut. Neither of these are vulnerabilities; to call them such does a disservice to your customer's understanding of technical language.
- borski 12y ago
  Sorry, perhaps it's a little unclear. Rudimentary Scan means you ran a scan from our homepage, pre-verification, and we couldn't run our full suite of modules. All that is saying is that now since you're verified you should re-run a scan which will look for quite a bit more. We figured putting it there would be the best way of not giving a false sense of security from our 'rudimentary scan.' We make it 'borderline unsafe' so as to prevent that false sense of security. I can totally understand your confusion, and we should make this clearer. Ideas / feedback are always appreciated. :) The email address is an informational vulnerability, which is dismissible. Does that clarify things? (cofounder of Tinfoil)
  - ctz 12y ago
    If I were you, I would put the full scan at the end of the setup workflow (straight after verification), so that by the time the user gets to their report, you have full results for them.
    - borski 12y ago
      Feedback appreciated. We're rethinking the whole flow, and we'll definitely try to make sure it's clearer. The intentions were good, I promise. :)
dopamean 12y ago
A semi-interesting, completely off topic note on something I just learned after walking the earth for 28 years. The word "primer" as it is used here is actually pronounced primmer and not PRIMEr. Apparently the origins of both words are pretty unrelated.
- borski 12y ago
  OED accepts both pronunciations. Apparently "primmer" is the British way of pronouncing it.
atmosx 12y ago
I'm not a security guru, but the most common scenario (even today, that big data and NoSQL/Graph dbs thrive) is a website running on a SQL (with some additional memcached/Redis or Mongo?). Say I encrypt the entries... Since the database runs in local host, if the host is compromised and subsequently 'rooted', what exactly does database encryption offer? The intruder will sniff/find the encryption key(s) since the key must be somewhere inside the application or in memory (don't know how, but I'm sure it's possible) in order to be able to decrypt data on the fly. The way I see it it's just added complexity for the admin with no gain. Is there any way of defending an SQL database even when the host is compromised?
- chacham15 12y ago
  Simplest solution to fix this is to change "Since the database runs in local host". Otherwise, it depends on the extent of the compromise. If the attacker got root access, then all of the data is compromised. Note: you can minimize the effects of this compromise by doing one of a few things: storing the hash of data (e.g. with passwords) so that the original data is still (relatively) safe, or you could also have the data encrypted with another key (e.g. if it is a credit card number) that does not reside on the local machine. If the attacker has not gotten root access, you can use different users to separate what potential data the attacker has access to.
  - pyre 12y ago
    I think that it's probably safer to assume that if the attacker controls a local user account, escalating that to root privs "isn't that difficult." I see such comments all of the time here by people that should know about security. Separating concerns be user account has its uses (e.g. sandboxing an app from accidentally screwing with files it doesn't need access to), but I don't think you necessarily want it as the linchpin of your encryption strategy.
    - chacham15 12y ago
      That is not a fair assumption. The reason being that you know nothing about the type of vulnerability that they are exploiting. For example, if the site has a SQL injection attack, user separation in the database would prove quite useful. > but I don't think you necessarily want it as the linchpin of your encryption strategy. Its not about having one thing at "the linchpin". In fact, you should not rely on any singular thing, but rather separate out pieces of information so that no part has more than it needs. The test user should not have access to the production database and vice versa. In sum, you should use multiple things to defend yourself, not just one.
deleted 12y ago
[deleted]

55 points by ShaneWilton 12y ago

> These functions belong to a family of functions known as key derivation functions: they use hashing to produce a digest which is suitable for use as a password hash. KDFs are not for storing password hashes, they are for key derivation. This is a subtle point, the main thing is to use KDFs as KDFs (to derive keys that are then used to encrypt data). Details in the first link here (the second is about common mistakes when using scrypt): - http://blog.ircmaxell.com/2014/03/why-i-dont-recommend-scrypt.html http://blog.ircmaxell.com/2014/03/why-i-dont-recommend-scryp... (bad title, he's referring to pass storage, not KDF usage) - http://vnhacker.blogspot.com/2014/04/fairy-tales-in-password-hashing-with.html http://vnhacker.blogspot.com/2014/04/fairy-tales-in-password... Another consideration is plausible deniability (PD) for situations where you're compelled to disclose your password(s). Our company writes Mac encryption software that specializes in this (and it uses scrypt). Here's a list of tools that offer PD (ours is called Espionage): https://en.wikipedia.org/wiki/Deniable_encryption#Software https://en.wikipedia.org/wiki/Deniable_encryption#Software I compared Espionage's PD to TrueCrypt's on reddit: http://www.reddit.com/r/security/comments/2b5icu/major_advancements_in_deniable_encryption_arrive/cj24a1n http://www.reddit.com/r/security/comments/2b5icu/major_advan...

ShaneWilton 12y ago
You're right, there's some ambiguous wording on my part. I'll be sure to make that section more precise right now. Looking back on that section, I should also be talking about work factors and such.
- itistoday2 12y ago
  My apologies, I wasn't right. Please see this reply chain: https://news.ycombinator.com/item?id=8088208 https://news.ycombinator.com/item?id=8088208
  - ShaneWilton 12y ago
    You're right about parts of the post being unclear. I'll be rewriting that paragraph to be more precise regardless.
tptacek 12y ago
The first of these two source suggests that with typical settings, due to a quirk with the way scrypt works, it's less secure than bcrypt. The second criticizes an API design with the scrypt code. Neither makes a case that password-based KDFs are "not for storing password hashes". Using a good password KDF as a password authenticator is a fine decision.
- itistoday2 12y ago
  > The second criticizes an API design with the scrypt code. Yes, I didn't claim otherwise. > Neither makes a case that password-based KDFs are "not for storing password hashes". The first one did quite explicitly for scrypt: "And that's why I don't recommend it for password storage." > Using a good password KDF as a password authenticator is a fine decision. Apparently not so for scrypt, and scrypt is the best KDF I'm aware of (heard something about yescrypt being better but haven't had a chance to look into it). I don't see why a KDF is needed for password storage. What's wrong with SHA256 + random salt? If for some reason you want a slow hash function, use the salt MOD some constant to do several rounds of SHA256 (though I don't see why that would be necessary).
  - tptacek 12y ago
    The last sentence of this comment is exactly why I called it out. If "for some reason" you want a slow hash function? You must have a slow hash function to store a password authenticator. The salt provides virtually no protection; it blocks a single corner-case exploit.
    - itistoday2 12y ago
      > You must have a slow hash function to store a password authenticator. The salt provides virtually no protection; it blocks a single corner-case exploit. You're referring to simple dictionary attacks on a large database of passwords? Even bcrypt won't help much with a password from the top 10k list. I wouldn't say that stopping rainbow tables is "virtually no protection". Please explain what you're referring to.

ctz 12y ago

Not related to the article, but the source: Having my email address on my blog is a vulnerability. Another vulnerability is named 'Rudimentary Scan', meaning 'we looked, and your site has no client-side vulnerabilities'. These issues mean my blog is 'borderline unsafe'. Wut. Neither of these are vulnerabilities; to call them such does a disservice to your customer's understanding of technical language.

borski 12y ago
Sorry, perhaps it's a little unclear. Rudimentary Scan means you ran a scan from our homepage, pre-verification, and we couldn't run our full suite of modules. All that is saying is that now since you're verified you should re-run a scan which will look for quite a bit more. We figured putting it there would be the best way of not giving a false sense of security from our 'rudimentary scan.' We make it 'borderline unsafe' so as to prevent that false sense of security. I can totally understand your confusion, and we should make this clearer. Ideas / feedback are always appreciated. :) The email address is an informational vulnerability, which is dismissible. Does that clarify things? (cofounder of Tinfoil)
- ctz 12y ago
  If I were you, I would put the full scan at the end of the setup workflow (straight after verification), so that by the time the user gets to their report, you have full results for them.
  - borski 12y ago
    Feedback appreciated. We're rethinking the whole flow, and we'll definitely try to make sure it's clearer. The intentions were good, I promise. :)

dopamean 12y ago

A semi-interesting, completely off topic note on something I just learned after walking the earth for 28 years. The word "primer" as it is used here is actually pronounced primmer and not PRIMEr. Apparently the origins of both words are pretty unrelated.

borski 12y ago
OED accepts both pronunciations. Apparently "primmer" is the British way of pronouncing it.

atmosx 12y ago

I'm not a security guru, but the most common scenario (even today, that big data and NoSQL/Graph dbs thrive) is a website running on a SQL (with some additional memcached/Redis or Mongo?). Say I encrypt the entries... Since the database runs in local host, if the host is compromised and subsequently 'rooted', what exactly does database encryption offer? The intruder will sniff/find the encryption key(s) since the key must be somewhere inside the application or in memory (don't know how, but I'm sure it's possible) in order to be able to decrypt data on the fly. The way I see it it's just added complexity for the admin with no gain. Is there any way of defending an SQL database even when the host is compromised?

chacham15 12y ago
Simplest solution to fix this is to change "Since the database runs in local host". Otherwise, it depends on the extent of the compromise. If the attacker got root access, then all of the data is compromised. Note: you can minimize the effects of this compromise by doing one of a few things: storing the hash of data (e.g. with passwords) so that the original data is still (relatively) safe, or you could also have the data encrypted with another key (e.g. if it is a credit card number) that does not reside on the local machine. If the attacker has not gotten root access, you can use different users to separate what potential data the attacker has access to.
- pyre 12y ago
  I think that it's probably safer to assume that if the attacker controls a local user account, escalating that to root privs "isn't that difficult." I see such comments all of the time here by people that should know about security. Separating concerns be user account has its uses (e.g. sandboxing an app from accidentally screwing with files it doesn't need access to), but I don't think you necessarily want it as the linchpin of your encryption strategy.
  - chacham15 12y ago
    That is not a fair assumption. The reason being that you know nothing about the type of vulnerability that they are exploiting. For example, if the site has a SQL injection attack, user separation in the database would prove quite useful. > but I don't think you necessarily want it as the linchpin of your encryption strategy. Its not about having one thing at "the linchpin". In fact, you should not rely on any singular thing, but rather separate out pieces of information so that no part has more than it needs. The test user should not have access to the production database and vice versa. In sum, you should use multiple things to defend yourself, not just one.

deleted 12y ago

[deleted]