10 ms·

> In general it is a bad idea to store secrets in the database itself Perhaps my ignorance is showing but do hashed passwords count as secrets? Because those a

> In general it is a bad idea to store secrets in the database itself Perhaps my ignorance is showing but do hashed passwords count as secrets? Because those are usually stored in databases.

orev 6y ago
Properly hashed/salted passwords are considered safe to store in a database. You still wouldn’t want them to be stolen, but they are not in plaintext. There are other times when you do need to store secrets that you need to be able to decrypt so you can use them.
- eyelidlessness 6y ago
  Nothing is considered safe, and everything is considered borrowed time if you're storing encrypted data. Retrievable data isn't any less encrypted than hashes, it's just a bidirectional encryption.
- ithkuil 6y ago
  It all depends on two factors: a) minimal required entropy of the passwords themselves. b) how computation expensive is the hash function itself. The threat model in password bashing scenario is that the attacker wants to discover the password and then use it to impersonate the user. Thus the attacker needs to guess the password, by searching though the space of all possible passwords (using heuristics such as dictionary, common letter/number replacements, common position of symbols mandatory in many password policies) and the the candidate password needs to be hashed and compared to the known good hash. By using an expensive hash function such as bcrypt you can limit the ability of exhaustively search the space of all possible passwords. But if a user uses a weak password, the attack will succeed and leaking the hashes will make the search go several orders of magnitude faster than online brute forcing (I.e real login attempts, which can be throttled)
  - e12e 6y ago
    c) the fact that in the real world, some users will re-use "secure" passwords.
    - ithkuil 6y ago
      In the real world this reduces password entropy, thus it falls under (a) :-) EDIT: perhaps it would be easier to just think of it in terms of predictability. Often tech jargon gets in the way.
      - e12e 6y ago
        Yes and no. You could argue the infamous Debian openssl bug reduced entropy [1], but it really is about compromised keys. Or in this case compromised passwords. [1] in a sense, this is of course obviously and literally true (that it is about entropy). But I think the more interesting distinction is between leaked secrets, and "bad" secrets - in that light the "Debian keys" all form a group of "leaked" secret keys. There's nothing inherently wrong with the generated keys - but you'd have to improbably unlucky to "accidentally" generate any given compromised key using a good rng. In the same way, the password "TGSCfkL$seG2@tn5DowiFJ$nPj%KW#vMT" that bitwarden just generated for me isn't inherently bad - until now, that it's been published/leaked. So I'm not sure I agree that it's all/only about entropy. I think it illustrates well that secrets and especially those managed by people (such as passwords and passphrases) have systemic component: the system seen as a whole incorporates users, other systems as well as any one given system (Gmail is affected by leaked Hotmail accounts, and vice-versa).
        ithkuil 6y ago
        I'm not disagreeing. I just wanted to highlight to to break a password you need a) to guess it b) to verify it. Leaking the hashes affects (b)
paulryanrogers 6y ago
Passwords usually get hashed in app layer so DB never has the key. In the same way encryption is usually done entirely outside DBs as they tend to leak or get breached. So the ciphertext without any potential access to keys is generally acceptable for storing in a DB, ideally isolated for one purpose.
- systemvoltage 6y ago
  This is the correct answer. Hashing can happen at many different layers in the app or even as a ephemeral Postgres function call in the database. Storing hashes is not storing secrets. If one has to worry about exposing the lock to public, then there is something wrong with the security of that lock. Use a better lock.
Nzen 6y ago
Presumably, one could retort that pragmatic practice isn't the optimal practice. Which is to say, if a service could derive authorization without secrets, that would present a smaller security attack surface than one that needed plain or hashed secrets (assuming a risk profile that includes data exfiltration). The service could, for example, outsource that to an oauth provider [0], albiet at the cost of accepting the provider's level of service as it's absolute maximum. [0] https://blog.codinghorror.com/the-god-login/ https://blog.codinghorror.com/the-god-login/
deleted 6y ago
[deleted]
fuoqi 6y ago
Ideally passwords should not leave user's computer. Use a strong password-hashing function (e.g. argon2) and a PAKE [0] protocol (e.g. OPAQUE) if you can. [0]: https://en.wikipedia.org/wiki/Password-authenticated_key_agreement https://en.wikipedia.org/wiki/Password-authenticated_key_agr...
tialaramex 6y ago
I see lots of people are arguing that it's not a secret. Here's a thought experiment to try if you're considering taking that side of the argument: If a web site you use offers all their password hashes as a free download, would their users be unaffected? Would you come to HN to write about how that's actually fine because those aren't secrets after all? To be clear there are credentials that work that way. The RSA key your browser is automatically using to check this is https://news.ycombinator.com/ https://news.ycombinator.com/ is public, it is right there for us all to see. Knowing it doesn't help you impersonate Hacker News. More easily compared to your password would be U2F/ WebAuthn credentials. Here are my 100% genuine WebAuthn credentials for a particular web site, Base64 encoded, this is what the site stores in its database to authenticate me: id: tmVUNeAST4JhA5HMN61Ddk1G0FVK+O8K3gY+/z/HE6WWyjny9cDCWY1LsmNcLP63qNdugU2itZRyBM1LNAJfFA1TZ1qxOoiqXMK7R3KqPg++UwIrdqdr0na4BWP2uPm1 public key: pQECAyYgASFYIGMcwjIjCbidUjT3hSEvZOAme++M+uQ1+I36am+GgiICIlgguso4MWdGHQRs82kcQJjaGN7Lf8NUHNYld+XktlD8hBg= You can't do anything with that, you can't even use it on your own web site to authenticate me, it's useless But passwords and even a password hash are not like that, they have to be kept secret or else an attacker gains a significant advantage.
- systemvoltage 6y ago
  And...you didn't explain the most important part of your ramble: > they have to be kept secret or else an attacker gains a significant advantage. Can you please explain? Why is this the case? Sorry to be snarky, but after reading your elaborate response, I was left thirsty.
  - tialaramex 6y ago
    Without the hashes an attacker's only way to find out the password for an account is to guess. They need to try logging into the account with a guessed password. Maybe they have a very good guess, and it'll be right first time, more likely they need a few tries, even thousands or millions of tries to find the right password. Every time they try, that's another opportunity to detect the attack, to stop them or slow them down. That makes the attack less likely to be ultimately successful. Even if the system doesn't go out of its way to try to do any of those things, chances are you just can't do millions of guesses this way in a reasonable time. But if they have a password hash for the account they can check as many guesses as they can afford compute power against that hash. Millions per second on a cheap home PC is common, maybe it'd be as low as thousands per second with the best possible choice of password hash, tuned to be aggressively expensive. Much more and "authentication" becomes annoyingly expensive for the legitimate system, so it is never done in practice. If the password to be guessed is ten random alphanumerics from a decent password manager then it doesn't matter, neither approach gets the attacker in. But while my passwords are like that, my mother's passwords certainly aren't and neither are those of most normal users.
Others 6y ago
I feel like hashed passwords are somewhere in between “completely okay to publish” and secret. With good crypto, it should be difficult to make a password hash useful even if it is public. (Although weak passwords can make this worse if salts are also made public.) However as part of a defense in depth strategy, you probably don’t want to release them publicly. I feel like there are different levels of secrecy at play here. I think the article means things more like ssh keys or api keys specifically.
- e12e 6y ago
  No, the purpose of the hash is to be able to verify a password. It has to be reasonably quick - it's in the login path. This is different from a public key - its unfeasible to derive a secret key from a public key. Its by design trivial to verify a correct password guess against a salted hash. In general, you won't have a lot of candidate secret keys to try against public keys - but all you need to get candidate passwords is to offer up a service that "check if your password is secure/compromised/etc". You might not crack root@box,but can easily verify that you have access to ceo@box...
aapeli 6y ago
Correctly hashed (with salt and a memory+time hard hash) passwords are taken to be brute-force hard to crack. In that sense it's as safe to publish such hashed passwords on the internet, in the same way a website's public key is published on the internet. In fact, it's good practice to set hash parameters such that it's slower to brute-force passwords than asymmetric keys (e.g. TLS certs). However, the big difference is that TLS private keys are randomly generated, and of a fixed length, whereas passwords are user chosen. So an attacker could do a dictionary attack and probably uncover a number of passwords using that (e.g. just try out "password" on all the hashed passwords). Hashed passwords are only as hard to crack as the passwords themselves.
- jopsen 6y ago
  Publishing a salted hash (with if it's memory+time hard hash) is the same as allowing unlimited login attempts. Limiting login attempts by ip, username, and time is the best way to mitigate attacks. Even a weak password is hard to crack with 5 attempts per day :)
  - hanche 6y ago
    Given a salted hash, you can test passwords many orders of magnitude faster than you can do online. As some attackers can control a botnet, limiting attempts by ip has limited value. If you limit by username and time, you open the door to a denial-of-service attack: I could lock you out of your account by simply trying to log in as you repeatedly. There are few easy answers in security.
michelpp 6y ago
Yes they are secret, which is why databases usually don't store them, they store a salted hash. The secret , the key in a sense, is in the users head, or password manager I guess these days. An attacker cannot get the password from a correctly hashed and encrypted password because that information is not in there. This is different than storing a key in the database that can turn encrypted data into unencrypted data. This is what I meant by not storing secrets. Instead of storing keys, pgsodium use encourages storing key ids (of a given length in a context). These keys can be used to reveal plaintext, so they shouldn't be stored but only derived. So the trick to avoiding storing keys is to store the key id and derive the key, being careful not to leak it in the logs. This can be done by passing it directly from pgsodium_derive() straight into a function that takes a key. I'm considering adding a "mirror" api that doesn't take bytea keys but instead only key ids and always derives. I'm still kicking that idea around. But your point is good, I should be less cavalier with my definitions of secret and key. I'm working on improvement for the "encrypted column" section that will explain more of this, and show how pgsodium can work in hand with Row Level Security. Stay tuned.
- Varriount 6y ago
  Ah, I should have added "salted" to my original description. What are your opinions of the `crypt` and `gen_salt` functions from the pgcrypto module[0]? https://www.postgresql.org/docs/current/pgcrypto.html#id-1.11.7.34.6 https://www.postgresql.org/docs/current/pgcrypto.html#id-1.1...
  - michelpp 6y ago
    They're just fine, they do the job. Note that in PostgreSQL 10 they updates the user password storage and challenge mechanism to SCRAM https://info.crunchydata.com/blog/how-to-upgrade-postgresql-passwords-to-scram https://info.crunchydata.com/blog/how-to-upgrade-postgresql-...

> In general it is a bad idea to store secrets in the database itself Perhaps my ignorance is showing but do hashed passwords count as secrets? Because those are usually stored in databases.

orev 6y ago

Properly hashed/salted passwords are considered safe to store in a database. You still wouldn’t want them to be stolen, but they are not in plaintext. There are other times when you do need to store secrets that you need to be able to decrypt so you can use them.

eyelidlessness 6y ago
Nothing is considered safe, and everything is considered borrowed time if you're storing encrypted data. Retrievable data isn't any less encrypted than hashes, it's just a bidirectional encryption.
ithkuil 6y ago
It all depends on two factors: a) minimal required entropy of the passwords themselves. b) how computation expensive is the hash function itself. The threat model in password bashing scenario is that the attacker wants to discover the password and then use it to impersonate the user. Thus the attacker needs to guess the password, by searching though the space of all possible passwords (using heuristics such as dictionary, common letter/number replacements, common position of symbols mandatory in many password policies) and the the candidate password needs to be hashed and compared to the known good hash. By using an expensive hash function such as bcrypt you can limit the ability of exhaustively search the space of all possible passwords. But if a user uses a weak password, the attack will succeed and leaking the hashes will make the search go several orders of magnitude faster than online brute forcing (I.e real login attempts, which can be throttled)
- e12e 6y ago
  c) the fact that in the real world, some users will re-use "secure" passwords.
  - ithkuil 6y ago
    In the real world this reduces password entropy, thus it falls under (a) :-) EDIT: perhaps it would be easier to just think of it in terms of predictability. Often tech jargon gets in the way.
    - e12e 6y ago
      Yes and no. You could argue the infamous Debian openssl bug reduced entropy [1], but it really is about compromised keys. Or in this case compromised passwords. [1] in a sense, this is of course obviously and literally true (that it is about entropy). But I think the more interesting distinction is between leaked secrets, and "bad" secrets - in that light the "Debian keys" all form a group of "leaked" secret keys. There's nothing inherently wrong with the generated keys - but you'd have to improbably unlucky to "accidentally" generate any given compromised key using a good rng. In the same way, the password "TGSCfkL$seG2@tn5DowiFJ$nPj%KW#vMT" that bitwarden just generated for me isn't inherently bad - until now, that it's been published/leaked. So I'm not sure I agree that it's all/only about entropy. I think it illustrates well that secrets and especially those managed by people (such as passwords and passphrases) have systemic component: the system seen as a whole incorporates users, other systems as well as any one given system (Gmail is affected by leaked Hotmail accounts, and vice-versa).
      - ithkuil 6y ago
        I'm not disagreeing. I just wanted to highlight to to break a password you need a) to guess it b) to verify it. Leaking the hashes affects (b)

paulryanrogers 6y ago

Passwords usually get hashed in app layer so DB never has the key. In the same way encryption is usually done entirely outside DBs as they tend to leak or get breached. So the ciphertext without any potential access to keys is generally acceptable for storing in a DB, ideally isolated for one purpose.

systemvoltage 6y ago
This is the correct answer. Hashing can happen at many different layers in the app or even as a ephemeral Postgres function call in the database. Storing hashes is not storing secrets. If one has to worry about exposing the lock to public, then there is something wrong with the security of that lock. Use a better lock.

Nzen 6y ago

Presumably, one could retort that pragmatic practice isn't the optimal practice. Which is to say, if a service could derive authorization without secrets, that would present a smaller security attack surface than one that needed plain or hashed secrets (assuming a risk profile that includes data exfiltration). The service could, for example, outsource that to an oauth provider [0], albiet at the cost of accepting the provider's level of service as it's absolute maximum. [0] https://blog.codinghorror.com/the-god-login/ https://blog.codinghorror.com/the-god-login/

deleted 6y ago

[deleted]

fuoqi 6y ago

Ideally passwords should not leave user's computer. Use a strong password-hashing function (e.g. argon2) and a PAKE [0] protocol (e.g. OPAQUE) if you can. [0]: https://en.wikipedia.org/wiki/Password-authenticated_key_agreement https://en.wikipedia.org/wiki/Password-authenticated_key_agr...

tialaramex 6y ago

I see lots of people are arguing that it's not a secret. Here's a thought experiment to try if you're considering taking that side of the argument: If a web site you use offers all their password hashes as a free download, would their users be unaffected? Would you come to HN to write about how that's actually fine because those aren't secrets after all? To be clear there are credentials that work that way. The RSA key your browser is automatically using to check this is https://news.ycombinator.com/ https://news.ycombinator.com/ is public, it is right there for us all to see. Knowing it doesn't help you impersonate Hacker News. More easily compared to your password would be U2F/ WebAuthn credentials. Here are my 100% genuine WebAuthn credentials for a particular web site, Base64 encoded, this is what the site stores in its database to authenticate me: id: tmVUNeAST4JhA5HMN61Ddk1G0FVK+O8K3gY+/z/HE6WWyjny9cDCWY1LsmNcLP63qNdugU2itZRyBM1LNAJfFA1TZ1qxOoiqXMK7R3KqPg++UwIrdqdr0na4BWP2uPm1 public key: pQECAyYgASFYIGMcwjIjCbidUjT3hSEvZOAme++M+uQ1+I36am+GgiICIlgguso4MWdGHQRs82kcQJjaGN7Lf8NUHNYld+XktlD8hBg= You can't do anything with that, you can't even use it on your own web site to authenticate me, it's useless But passwords and even a password hash are not like that, they have to be kept secret or else an attacker gains a significant advantage.

systemvoltage 6y ago
And...you didn't explain the most important part of your ramble: > they have to be kept secret or else an attacker gains a significant advantage. Can you please explain? Why is this the case? Sorry to be snarky, but after reading your elaborate response, I was left thirsty.
- tialaramex 6y ago
  Without the hashes an attacker's only way to find out the password for an account is to guess. They need to try logging into the account with a guessed password. Maybe they have a very good guess, and it'll be right first time, more likely they need a few tries, even thousands or millions of tries to find the right password. Every time they try, that's another opportunity to detect the attack, to stop them or slow them down. That makes the attack less likely to be ultimately successful. Even if the system doesn't go out of its way to try to do any of those things, chances are you just can't do millions of guesses this way in a reasonable time. But if they have a password hash for the account they can check as many guesses as they can afford compute power against that hash. Millions per second on a cheap home PC is common, maybe it'd be as low as thousands per second with the best possible choice of password hash, tuned to be aggressively expensive. Much more and "authentication" becomes annoyingly expensive for the legitimate system, so it is never done in practice. If the password to be guessed is ten random alphanumerics from a decent password manager then it doesn't matter, neither approach gets the attacker in. But while my passwords are like that, my mother's passwords certainly aren't and neither are those of most normal users.

Others 6y ago

I feel like hashed passwords are somewhere in between “completely okay to publish” and secret. With good crypto, it should be difficult to make a password hash useful even if it is public. (Although weak passwords can make this worse if salts are also made public.) However as part of a defense in depth strategy, you probably don’t want to release them publicly. I feel like there are different levels of secrecy at play here. I think the article means things more like ssh keys or api keys specifically.

e12e 6y ago
No, the purpose of the hash is to be able to verify a password. It has to be reasonably quick - it's in the login path. This is different from a public key - its unfeasible to derive a secret key from a public key. Its by design trivial to verify a correct password guess against a salted hash. In general, you won't have a lot of candidate secret keys to try against public keys - but all you need to get candidate passwords is to offer up a service that "check if your password is secure/compromised/etc". You might not crack root@box,but can easily verify that you have access to ceo@box...

aapeli 6y ago

Correctly hashed (with salt and a memory+time hard hash) passwords are taken to be brute-force hard to crack. In that sense it's as safe to publish such hashed passwords on the internet, in the same way a website's public key is published on the internet. In fact, it's good practice to set hash parameters such that it's slower to brute-force passwords than asymmetric keys (e.g. TLS certs). However, the big difference is that TLS private keys are randomly generated, and of a fixed length, whereas passwords are user chosen. So an attacker could do a dictionary attack and probably uncover a number of passwords using that (e.g. just try out "password" on all the hashed passwords). Hashed passwords are only as hard to crack as the passwords themselves.

jopsen 6y ago
Publishing a salted hash (with if it's memory+time hard hash) is the same as allowing unlimited login attempts. Limiting login attempts by ip, username, and time is the best way to mitigate attacks. Even a weak password is hard to crack with 5 attempts per day :)
- hanche 6y ago
  Given a salted hash, you can test passwords many orders of magnitude faster than you can do online. As some attackers can control a botnet, limiting attempts by ip has limited value. If you limit by username and time, you open the door to a denial-of-service attack: I could lock you out of your account by simply trying to log in as you repeatedly. There are few easy answers in security.

michelpp 6y ago

Yes they are secret, which is why databases usually don't store them, they store a salted hash. The secret , the key in a sense, is in the users head, or password manager I guess these days. An attacker cannot get the password from a correctly hashed and encrypted password because that information is not in there. This is different than storing a key in the database that can turn encrypted data into unencrypted data. This is what I meant by not storing secrets. Instead of storing keys, pgsodium use encourages storing key ids (of a given length in a context). These keys can be used to reveal plaintext, so they shouldn't be stored but only derived. So the trick to avoiding storing keys is to store the key id and derive the key, being careful not to leak it in the logs. This can be done by passing it directly from pgsodium_derive() straight into a function that takes a key. I'm considering adding a "mirror" api that doesn't take bytea keys but instead only key ids and always derives. I'm still kicking that idea around. But your point is good, I should be less cavalier with my definitions of secret and key. I'm working on improvement for the "encrypted column" section that will explain more of this, and show how pgsodium can work in hand with Row Level Security. Stay tuned.

Varriount 6y ago
Ah, I should have added "salted" to my original description. What are your opinions of the `crypt` and `gen_salt` functions from the pgcrypto module[0]? https://www.postgresql.org/docs/current/pgcrypto.html#id-1.11.7.34.6 https://www.postgresql.org/docs/current/pgcrypto.html#id-1.1...
- michelpp 6y ago
  They're just fine, they do the job. Note that in PostgreSQL 10 they updates the user password storage and challenge mechanism to SCRAM https://info.crunchydata.com/blog/how-to-upgrade-postgresql-passwords-to-scram https://info.crunchydata.com/blog/how-to-upgrade-postgresql-...