4 ms·

Serious question: Why isn't there a "best practice" for this? A library everybody uses to store passwords in the most secure way possible, and when more secure

by nsxwolf 11y ago

Serious question: Why isn't there a "best practice" for this? A library everybody uses to store passwords in the most secure way possible, and when more secure ways are found, the library is updated and not your code? Everything is so low level, manual and error prone for such a basic part of app security.

IgorPartola 11y ago
Django provides this and you don't have to worry about it. However, they did not for a bit, even as the discussions raged on about how salt + SHA1 is bad. The reason was that Django is 100% pure Python and at the time the only available implementations of any of this were C Python extensions. Ideally, I'd love to see bcrypt, scrypt, and PBKDF2 included directly in Python's stdlib so that people wouldn't have to re-invent the wheel every time they use the language.
- masklinn 11y ago
  pbkdf2 is included in Python's stdlib since 2.7.8[0] and 3.4[1], though sadly the interface isn't great: it outputs the hash only, not a modular crypt[2] and it requires a salt instead of generating one. Passlib[3] provides more algorithms, a much better interface and handles hash migrations[4] ridiculously cleanly. [0] https://docs.python.org/2/library/hashlib.html#hashlib.pbkdf2_hmac https://docs.python.org/2/library/hashlib.html#hashlib.pbkdf... [1] https://docs.python.org/3.5/library/hashlib.html#hashlib.pbkdf2_hmac https://docs.python.org/3.5/library/hashlib.html#hashlib.pbk... [2] http://pythonhosted.org/passlib/modular_crypt_format.html#modular-crypt-format http://pythonhosted.org/passlib/modular_crypt_format.html#mo... [3] http://pythonhosted.org/passlib http://pythonhosted.org/passlib [4] increase of rounds # or KDF change http://pythonhosted.org/passlib/lib/passlib.context-tutorial.html#deprecation-hash-migration http://pythonhosted.org/passlib/lib/passlib.context-tutorial...
mkozlows 11y ago
The library can't be updated automatically, because suddenly all your stored passwords wouldn't work. If it's a big monolithic library like Spring, they just deprecate APIs and add new ones (which is in fact what they do); if you're in a microframework language like Node, you're on your own to rip out your md5 module and switch it to bcrypt.
- masklinn 11y ago
  > The library can't be updated automatically Why not? > because suddenly all your stored passwords wouldn't work. Passlib handles that with a concept of "deprecated algorithms"[0]: a cryptcontext has a list of algorithms it can accept as inputs to validate against, and a subset of these can be marked as deprecated[1]. The passwords hashed with deprecated algorithms will validate but they'll be flagged as needing an upgrade. The `verify_and_upgrade` method will return both the validity of the password and a new hash if it should be upgraded, so the normal pattern is: valid, new_hash = pass_ctx.verify_and_update(password, old_hash) if valid: if new_hash: # store new hash for user # password was valid else: # password wasn't valid [0] http://pythonhosted.org/passlib/lib/passlib.context-tutorial.html#deprecation-hash-migration http://pythonhosted.org/passlib/lib/passlib.context-tutorial... [1] in recent versions you can even set the deprecated list to "auto", this will consider all algorithms but the default (the one used if you encrypt with the cryptcontext without requesting anything specific) to be deprecated. It's a thing of beauty.
- ubernostrum 11y ago
  Django's auth system handles updates safely and cleanly. Don't know about others, but the mechanism is simple: * The value stored in the DB is actually of the form "algorithm$iterations$salt$hash". * The current preferred algorithm and iterations are configurable in the settings. * When logging a user in, the algorithm/iterations in the stored value are used to check the password. * If the stored value's algorithm/iterations are out of sync with the preferred algorithm/iterations in the settings, then the last step of the login process is to re-hash the (verified correct) submitted password with the new preferred algorithm/iterations, and push that value back to the DB. Which creates a rolling refresh of the stored values as users log in, which in turn lets you gracefully change the algorithm or number of iterations.
yeukhon 11y ago
PBKDF2, bcrypt and scrypt actually have their own advantages and disadvantages. I think for embedded devices PBKDF2 is preferable and more performant, and given different threat model, you may choose one over the other. A classic read is https://blog.mozilla.org/warner/2014/05/23/the-new-sync-protocol/ https://blog.mozilla.org/warner/2014/05/23/the-new-sync-prot... (unfortunately the presentation is 404) but the blog post is enough for a read.
lotyrin 11y ago
Because of all the developers that consume open source code to provide value to their organizations, the proportion who are willing, able and whose organizations allow them (not just permission but resources) to contribute back is effectively zero. Also all the tools required to do so still suck, for the same reason. For just one example, why - when there's billions of humans that each have hundreds of data points which are relevant to their lives - do we only have maybe hundreds of datastores, reporting tools or visualization tools? Hint: it's not because of the universal applicability and quality of existing products. Humans aren't very good at considering others or planning very far ahead. On top of that we're not very good yet at training people who can code despite the acute demand for them as everyone transitions paper processes to digital or old digital processes to new digital processes. Everyone is too busy building yesterday to worry about tomorrow, even on small scales. It seems only outlier organizations have the discipline and foresight to invest in even private internal tools or processes, let alone public ones.
dfabulich 11y ago
There are best practices, but they're written documents, not libraries. https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet Why not? 1) Language fragmentation. Java developers aren't going to use a C# library or a PHP library. So instead we're stuck with, "If you're on Python, use X; if you're on Ruby, use Y..." 2) Password storage is a problem that's easy to get wrong without knowing it. If you have a DIY password storage system that accepts valid passwords, rejects invalid passwords, and looks scrambled when you view them in the database, it's hard for non-experts to notice that it's broken in any other way. 3) The combination of those two problems results makes it difficult for the security community to reach consensus on good language-specific libraries. Competent Java security experts can't easily evaluate a PHP scrypt library. So all they can say is, "allow long passwords, use per-password salts, and use PBKDF2/scrypt/bcrypt with a pretty big work factor, but don't do it yourself."
- swsieber 11y ago
  With HaXe though you might be able to swing something like this. In fact something like passlib (python) would be beautiful but dangerous - imagine cross language bugs ( ie the same bug in Java, C++, Python, etc.)
  - voltagex_ 11y ago
    I wonder if in large organisations it'd be better to have a single service handling password management (with split databases still). That way you could lock it down to one set of IPs, all access monitored and more importantly one password storage implementation.
    - mschuster91 11y ago
      Such a thing exists, it's called LDAP. Bonus effect, many LDAP implementations (Windows AD, OpenLDAP, Lotus) give you replication and failover for free.
      - 11y ago

by nsxwolf 11y ago

IgorPartola 11y ago

Django provides this and you don't have to worry about it. However, they did not for a bit, even as the discussions raged on about how salt + SHA1 is bad. The reason was that Django is 100% pure Python and at the time the only available implementations of any of this were C Python extensions. Ideally, I'd love to see bcrypt, scrypt, and PBKDF2 included directly in Python's stdlib so that people wouldn't have to re-invent the wheel every time they use the language.

masklinn 11y ago
pbkdf2 is included in Python's stdlib since 2.7.8[0] and 3.4[1], though sadly the interface isn't great: it outputs the hash only, not a modular crypt[2] and it requires a salt instead of generating one. Passlib[3] provides more algorithms, a much better interface and handles hash migrations[4] ridiculously cleanly. [0] https://docs.python.org/2/library/hashlib.html#hashlib.pbkdf2_hmac https://docs.python.org/2/library/hashlib.html#hashlib.pbkdf... [1] https://docs.python.org/3.5/library/hashlib.html#hashlib.pbkdf2_hmac https://docs.python.org/3.5/library/hashlib.html#hashlib.pbk... [2] http://pythonhosted.org/passlib/modular_crypt_format.html#modular-crypt-format http://pythonhosted.org/passlib/modular_crypt_format.html#mo... [3] http://pythonhosted.org/passlib http://pythonhosted.org/passlib [4] increase of rounds # or KDF change http://pythonhosted.org/passlib/lib/passlib.context-tutorial.html#deprecation-hash-migration http://pythonhosted.org/passlib/lib/passlib.context-tutorial...

mkozlows 11y ago

The library can't be updated automatically, because suddenly all your stored passwords wouldn't work. If it's a big monolithic library like Spring, they just deprecate APIs and add new ones (which is in fact what they do); if you're in a microframework language like Node, you're on your own to rip out your md5 module and switch it to bcrypt.

masklinn 11y ago
> The library can't be updated automatically Why not? > because suddenly all your stored passwords wouldn't work. Passlib handles that with a concept of "deprecated algorithms"[0]: a cryptcontext has a list of algorithms it can accept as inputs to validate against, and a subset of these can be marked as deprecated[1]. The passwords hashed with deprecated algorithms will validate but they'll be flagged as needing an upgrade. The `verify_and_upgrade` method will return both the validity of the password and a new hash if it should be upgraded, so the normal pattern is: valid, new_hash = pass_ctx.verify_and_update(password, old_hash) if valid: if new_hash: # store new hash for user # password was valid else: # password wasn't valid [0] http://pythonhosted.org/passlib/lib/passlib.context-tutorial.html#deprecation-hash-migration http://pythonhosted.org/passlib/lib/passlib.context-tutorial... [1] in recent versions you can even set the deprecated list to "auto", this will consider all algorithms but the default (the one used if you encrypt with the cryptcontext without requesting anything specific) to be deprecated. It's a thing of beauty.
ubernostrum 11y ago
Django's auth system handles updates safely and cleanly. Don't know about others, but the mechanism is simple: * The value stored in the DB is actually of the form "algorithm$iterations$salt$hash". * The current preferred algorithm and iterations are configurable in the settings. * When logging a user in, the algorithm/iterations in the stored value are used to check the password. * If the stored value's algorithm/iterations are out of sync with the preferred algorithm/iterations in the settings, then the last step of the login process is to re-hash the (verified correct) submitted password with the new preferred algorithm/iterations, and push that value back to the DB. Which creates a rolling refresh of the stored values as users log in, which in turn lets you gracefully change the algorithm or number of iterations.

yeukhon 11y ago

PBKDF2, bcrypt and scrypt actually have their own advantages and disadvantages. I think for embedded devices PBKDF2 is preferable and more performant, and given different threat model, you may choose one over the other. A classic read is https://blog.mozilla.org/warner/2014/05/23/the-new-sync-protocol/ https://blog.mozilla.org/warner/2014/05/23/the-new-sync-prot... (unfortunately the presentation is 404) but the blog post is enough for a read.

lotyrin 11y ago

Because of all the developers that consume open source code to provide value to their organizations, the proportion who are willing, able and whose organizations allow them (not just permission but resources) to contribute back is effectively zero. Also all the tools required to do so still suck, for the same reason. For just one example, why - when there's billions of humans that each have hundreds of data points which are relevant to their lives - do we only have maybe hundreds of datastores, reporting tools or visualization tools? Hint: it's not because of the universal applicability and quality of existing products. Humans aren't very good at considering others or planning very far ahead. On top of that we're not very good yet at training people who can code despite the acute demand for them as everyone transitions paper processes to digital or old digital processes to new digital processes. Everyone is too busy building yesterday to worry about tomorrow, even on small scales. It seems only outlier organizations have the discipline and foresight to invest in even private internal tools or processes, let alone public ones.

dfabulich 11y ago

There are best practices, but they're written documents, not libraries. https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet Why not? 1) Language fragmentation. Java developers aren't going to use a C# library or a PHP library. So instead we're stuck with, "If you're on Python, use X; if you're on Ruby, use Y..." 2) Password storage is a problem that's easy to get wrong without knowing it. If you have a DIY password storage system that accepts valid passwords, rejects invalid passwords, and looks scrambled when you view them in the database, it's hard for non-experts to notice that it's broken in any other way. 3) The combination of those two problems results makes it difficult for the security community to reach consensus on good language-specific libraries. Competent Java security experts can't easily evaluate a PHP scrypt library. So all they can say is, "allow long passwords, use per-password salts, and use PBKDF2/scrypt/bcrypt with a pretty big work factor, but don't do it yourself."

swsieber 11y ago
With HaXe though you might be able to swing something like this. In fact something like passlib (python) would be beautiful but dangerous - imagine cross language bugs ( ie the same bug in Java, C++, Python, etc.)
- voltagex_ 11y ago
  I wonder if in large organisations it'd be better to have a single service handling password management (with split databases still). That way you could lock it down to one set of IPs, all access monitored and more importantly one password storage implementation.
  - mschuster91 11y ago
    Such a thing exists, it's called LDAP. Bonus effect, many LDAP implementations (Windows AD, OpenLDAP, Lotus) give you replication and failover for free.
    - 11y ago