5 ms·

> It is really not difficult to use BCrypt Something I once saw in production: An otherwise normal usage of bcrypt, that "prehashed" the password with sha512

by bhaney 5y ago

> It is really not difficult to use BCrypt Something I once saw in production: An otherwise normal usage of bcrypt, that "prehashed" the password with sha512 before passing the resulting hash to bcrypt. This is conceptually reasonable (although not very useful) for reasons I don't think are worth going into, but had two critical problems. 1. It technically broke bcrypt's side-channel protection against timing attacks (bcrypt runs in constant time, while sha512 does not). 2. The much worse issue was that the binary hash was being passed to bcrypt. If bcrypt encounters a null byte in the input, it considers that to be the end of the input and ignores everything after, c-string style. Every byte of the binary hash had a 1/256 chance of being a null byte, which means 1/256 passwords result in a hash with the first byte being null, which means bcrypt will accept any of them as equivalent to the others. You could login to 0.4% of accounts by guessing ~256 completely random passwords (and a larger number of passwords were similarly but less extremely weakened) You can say "well that's their fault for mucking with the input to bcrypt" but I bet most people would believe that kind of operation was safe before it's explained to them why it's not. It's very easy to accidentally destroy the security of a crypto scheme without knowing it, and thinking that what you're doing is perfectly run of the mill and safe.

arcbyte 5y ago
Thank you for sharing that experience. It was a very interesting read. However, lest anyone get the wrong opinion, password hashing is a very simple, straightforward, solved problem and has been for years. One person doing something monumentally stupid doesn't make Auth some kind of cryptic minefield.
- deleted 5y ago
  [deleted]
- capableweb 5y ago
  > Stop trying to make something sound difficult that isn't. Password hashing is a very simple, straightforward, solved problem and has been for years. Try asking the following question on Twitter: "While passing binary data, which example is safer for storing passwords? [ ] $password | sha256 | bcrypt [ ] $password | bcrypt" What do you think the average programmer would say? Most would probably say they are either equal, or the first one, without knowing this specific thing, because most people don't implement their own password-hashing, they use library/framework provided ways that has been established as best practice already. But, can't blame them really, the difference is marginal and innocent on the surface, but once you understand the implementation, you'll see the holes.
  - whakim 5y ago
    I think the question isn't particularly relevant because many programmers are just using bcrypt in a simple way that's very well documented. I think it's reasonable to expect people to know that "auth is dangerous, don't just do your own special thing and assume it'll be fine." If you want to do something that diverges from the simple case, then please don't try to roll your own auth.
- bhaney 5y ago
  My point is precisely that it wasn't "monumentally stupid," but rather a pretty reasonable decision given what the average person would expect bcrypt to provide functionally. The behavior I described is an unexpected landmine that people using bcrypt need to be aware of. There are likely others, though probably not enough that I would describe it as a minefield. I would wager that most people reading this, if earnestly asked by a coworker "do you think it's fine for me to hash a password before passing it to bcrypt? I want to be able to support passwords over 72 characters and bcrypt truncates its input." would answer something along the lines of "I don't see how it could hurt" rather than "that's dangerous because a binary hash would result in a large portion of the passwords being hashed as an empty string" The engineers that originally implemented and reviewed this were not idiots, they just weren't security experts.
  - nicoburns 5y ago
    My answer would definitely be: I don't know, but why bother. We know bcrypt by itself works. We don't know whether it works combined with another hash, and it's extra code we need to maintain.
- wieghant 5y ago
  It's monumentally stupid to think auth is easy. Yes there are standard cookbooks and checklists out there. The hubris to think that auth fails due to a crypto scheme choice is why average programmers consistently fail at it. The yearly report of leaks in Fortune 500 companies should be proof enough of this. EDIT: To elaborate. Crypto scheme is only one tiny facet of a successful authentication solution. Where do you store the hash? What language and stack are you using? What is the maturity of libraries available to you? What protocols? And many more seemingly tiny decisions. All it takes is a lazy developer that imports an insecure transient dependency or snoozes on a CVE.
kelnos 5y ago
> You can say "well that's their fault for mucking with the input to bcrypt" but I bet most people would believe that kind of operation was safe before it's explained to them why it's not. What makes you think this foolish developer would not do the exact same thing before sending the data to an externally hosted auth service?
- GormanFletcher 5y ago
  The external auth service's API would receive textual input (expecting a string read from a textbox), not binary. A hash that included null bytes would output those in hex as ASCII zeroes, which wouldn't cause the same problem when the external service received the value.
  - jlokier 5y ago
    And that is, in fact, how you should perform a hash before bcrypt as well. bcrypt accepts text input, not binary. So if you use a pre-hash, you pass the hex or base64 output from the hash to bcrypt.
jlokier 5y ago
It's not that non-standard. You are describing the method Dropbox wrote a post about some years ago: https://dropbox.tech/security/how-dropbox-securely-stores-your-passwords https://dropbox.tech/security/how-dropbox-securely-stores-yo... > First, the plaintext password is transformed into a hash value using SHA512. This addresses two particular issues with bcrypt. Some implementations of bcrypt truncate the input to 72 bytes, which reduces the entropy of the passwords. Other implementations don’t truncate the input and are therefore vulnerable to DoS attacks because they allow the input of arbitrarily long passwords. By applying SHA, we can quickly convert really long passwords into a fixed length 512 bit value, solving both problems. > Next, this SHA512 hash is hashed again using bcrypt with a cost of 10, and a unique, per-user salt. Unlike cryptographic hash functions like SHA, bcrypt is designed to be slow and hard to speed up via custom hardware and GPUs. A work factor of 10 translates into roughly 100ms for all these steps on our servers. > Finally, the resulting bcrypt hash is encrypted with AES256 using a secret key (common to all hashes) that we refer to as a pepper. The pepper is a defense in depth measure. The pepper value is stored separately in a manner that makes it difficult to discover by an attacker (i.e. not in a database table). As a result, if only the password storage is compromised, the password hashes are encrypted and of no use to an attacker.
jlokier 5y ago
Your point 1 seems wrong. SHA-512 is constant time for any reasonable password length and any reasonable implementation. It hashes 1024-bit blocks in constant time, as it only uses 64-bit word operations such as add, shift, xor etc in a fixed order with fixed constants. There are no input data-dependent lookups, or other variable-time operations. If you are considering the string copy that happens when the unhashed password is copied into the block to be variable time, probably you shouldn't count that against SHA-512 because there will be variable-length string operations before bcrypt starts its hashing too. Not to mention parsing it out of HTTP parameters or reading it from a GUI control. (If you really care about timing observations, perhaps you'd pre-hash the password on the client so what is transmitted is fixed-length). Point 2 is of course a real problem. Though to be honest I've always regarded the NUL check and (sometimes, implementation-dependent) maximum length as unnecessary, arbitrary flaws in bcrypt - so bcrypt gets the blame. On grounds of purity and doing the right thing, I refuse to use raw bcrypt, and in fact I use: SHA-512 -> Base64 -> bcrypt when password hashing. Does that make me wrong? Only if Dropbox is wrong (I hope they just forgot to mention Base64 in their old security article.)
- bhaney 5y ago
  > Your point 1 seems wrong. SHA-512 is constant time for any reasonable password length and any reasonable implementation. I'm aware of this, and it's why there's a "technically" in my original point. Of course in practice it makes no difference, and I consider the better use of bcrypt's input space when prehashing to outweigh leaking a pretty useless amount of side-channel information for extraordinarily long passwords. Though as a follow up anecdote, one of the engineers there had set a 300kB password to test the prehashing implementation, and just continued to use it (via password manager). He was potentially leaking a bit of timing information. I'm not even sure if a password that long made a timing difference that was detectable through network jitter, but if it did and some hypothetical attacker was able to learn that his password was hundreds of kilobytes long, well... good luck. > in fact I use: SHA-512 -> Base64 -> bcrypt when password hashing. Does that make me wrong? No, using a base64 representation of the sha hash makes your scheme perfectly reasonable. Also note that the base64 representation of sha512 is a bit longer than the typical bcrypt input length, so you're shrinking the collision space somewhat by truncating your sha hash. Certainly not enough that it would matter at all, just a point I find interesting. Hypothetically, a binary representation with a pass that removed or replaced NUL bytes would preserve more entropy, but we're well past the point where it matters and would be playing with fire to consider that complexity.
deckard1 5y ago
What's been interesting to me is that during tech interviews, we eagerly have applicants roll their own sorting, or searching, or caching. But you never have questions about crypto or auth. Why one and not the other? Are both not critical to get right? And then I'm reminded of the bug that was hidden in plain sight for 20 years in Programming Pearls[1] in their implementation of a binary search. This same algorithm was copied in Java and elsewhere, near verbatim. So, yeah, maybe don't roll your own searching either. The experts have enough difficulty. Frankly, I don't trust most programmers to get anything right. People toss around "battle-tested" libraries and we all just take it for granted that that's what they are. But I recently found out one incredibly popular package for one incredibly popular language is doing something fairly dumb. Not insecure (that I can tell). Just simply unnecessary. They are signing session cookies. For reasons I could not fathom. Signing doesn't prevent cookie theft. It's just another layer that someone thought would add security over just sending a random string (hopefully your system/language's random generator is actually random). And that's how a lot of these mistakes happen. Let's add sha512 on top of our crypto. Because more = better, right? Speaking of random, there was another popular package out there for generating UUIDs that has a github issue where someone discovered the package was generating collisions. Yikes. [1] https://ai.googleblog.com/2006/06/extra-extra-read-all-about-it-nearly.html https://ai.googleblog.com/2006/06/extra-extra-read-all-about...
- yoyonamite 5y ago
  Isn't signing the session cookie to prevent cookie tampering? You can send a random string, but you would need to persist the random string somewhere in order to verify against it later
  - DougN7 5y ago
    The cookie value has to be stored in some sort of map\dictionary that links to the session data. If the cookie value is a random string, you don’t need to store any more data than you would with a hashed cookie value. Or maybe I’m misunderstanding your point?
    - 5y ago

by bhaney 5y ago

arcbyte 5y ago

Thank you for sharing that experience. It was a very interesting read. However, lest anyone get the wrong opinion, password hashing is a very simple, straightforward, solved problem and has been for years. One person doing something monumentally stupid doesn't make Auth some kind of cryptic minefield.

deleted 5y ago
[deleted]
capableweb 5y ago
> Stop trying to make something sound difficult that isn't. Password hashing is a very simple, straightforward, solved problem and has been for years. Try asking the following question on Twitter: "While passing binary data, which example is safer for storing passwords? [ ] $password | sha256 | bcrypt [ ] $password | bcrypt" What do you think the average programmer would say? Most would probably say they are either equal, or the first one, without knowing this specific thing, because most people don't implement their own password-hashing, they use library/framework provided ways that has been established as best practice already. But, can't blame them really, the difference is marginal and innocent on the surface, but once you understand the implementation, you'll see the holes.
- whakim 5y ago
  I think the question isn't particularly relevant because many programmers are just using bcrypt in a simple way that's very well documented. I think it's reasonable to expect people to know that "auth is dangerous, don't just do your own special thing and assume it'll be fine." If you want to do something that diverges from the simple case, then please don't try to roll your own auth.
bhaney 5y ago
My point is precisely that it wasn't "monumentally stupid," but rather a pretty reasonable decision given what the average person would expect bcrypt to provide functionally. The behavior I described is an unexpected landmine that people using bcrypt need to be aware of. There are likely others, though probably not enough that I would describe it as a minefield. I would wager that most people reading this, if earnestly asked by a coworker "do you think it's fine for me to hash a password before passing it to bcrypt? I want to be able to support passwords over 72 characters and bcrypt truncates its input." would answer something along the lines of "I don't see how it could hurt" rather than "that's dangerous because a binary hash would result in a large portion of the passwords being hashed as an empty string" The engineers that originally implemented and reviewed this were not idiots, they just weren't security experts.
- nicoburns 5y ago
  My answer would definitely be: I don't know, but why bother. We know bcrypt by itself works. We don't know whether it works combined with another hash, and it's extra code we need to maintain.
wieghant 5y ago
It's monumentally stupid to think auth is easy. Yes there are standard cookbooks and checklists out there. The hubris to think that auth fails due to a crypto scheme choice is why average programmers consistently fail at it. The yearly report of leaks in Fortune 500 companies should be proof enough of this. EDIT: To elaborate. Crypto scheme is only one tiny facet of a successful authentication solution. Where do you store the hash? What language and stack are you using? What is the maturity of libraries available to you? What protocols? And many more seemingly tiny decisions. All it takes is a lazy developer that imports an insecure transient dependency or snoozes on a CVE.

kelnos 5y ago

> You can say "well that's their fault for mucking with the input to bcrypt" but I bet most people would believe that kind of operation was safe before it's explained to them why it's not. What makes you think this foolish developer would not do the exact same thing before sending the data to an externally hosted auth service?

GormanFletcher 5y ago
The external auth service's API would receive textual input (expecting a string read from a textbox), not binary. A hash that included null bytes would output those in hex as ASCII zeroes, which wouldn't cause the same problem when the external service received the value.
- jlokier 5y ago
  And that is, in fact, how you should perform a hash before bcrypt as well. bcrypt accepts text input, not binary. So if you use a pre-hash, you pass the hex or base64 output from the hash to bcrypt.

jlokier 5y ago

It's not that non-standard. You are describing the method Dropbox wrote a post about some years ago: https://dropbox.tech/security/how-dropbox-securely-stores-your-passwords https://dropbox.tech/security/how-dropbox-securely-stores-yo... > First, the plaintext password is transformed into a hash value using SHA512. This addresses two particular issues with bcrypt. Some implementations of bcrypt truncate the input to 72 bytes, which reduces the entropy of the passwords. Other implementations don’t truncate the input and are therefore vulnerable to DoS attacks because they allow the input of arbitrarily long passwords. By applying SHA, we can quickly convert really long passwords into a fixed length 512 bit value, solving both problems. > Next, this SHA512 hash is hashed again using bcrypt with a cost of 10, and a unique, per-user salt. Unlike cryptographic hash functions like SHA, bcrypt is designed to be slow and hard to speed up via custom hardware and GPUs. A work factor of 10 translates into roughly 100ms for all these steps on our servers. > Finally, the resulting bcrypt hash is encrypted with AES256 using a secret key (common to all hashes) that we refer to as a pepper. The pepper is a defense in depth measure. The pepper value is stored separately in a manner that makes it difficult to discover by an attacker (i.e. not in a database table). As a result, if only the password storage is compromised, the password hashes are encrypted and of no use to an attacker.

jlokier 5y ago

Your point 1 seems wrong. SHA-512 is constant time for any reasonable password length and any reasonable implementation. It hashes 1024-bit blocks in constant time, as it only uses 64-bit word operations such as add, shift, xor etc in a fixed order with fixed constants. There are no input data-dependent lookups, or other variable-time operations. If you are considering the string copy that happens when the unhashed password is copied into the block to be variable time, probably you shouldn't count that against SHA-512 because there will be variable-length string operations before bcrypt starts its hashing too. Not to mention parsing it out of HTTP parameters or reading it from a GUI control. (If you really care about timing observations, perhaps you'd pre-hash the password on the client so what is transmitted is fixed-length). Point 2 is of course a real problem. Though to be honest I've always regarded the NUL check and (sometimes, implementation-dependent) maximum length as unnecessary, arbitrary flaws in bcrypt - so bcrypt gets the blame. On grounds of purity and doing the right thing, I refuse to use raw bcrypt, and in fact I use: SHA-512 -> Base64 -> bcrypt when password hashing. Does that make me wrong? Only if Dropbox is wrong (I hope they just forgot to mention Base64 in their old security article.)

bhaney 5y ago
> Your point 1 seems wrong. SHA-512 is constant time for any reasonable password length and any reasonable implementation. I'm aware of this, and it's why there's a "technically" in my original point. Of course in practice it makes no difference, and I consider the better use of bcrypt's input space when prehashing to outweigh leaking a pretty useless amount of side-channel information for extraordinarily long passwords. Though as a follow up anecdote, one of the engineers there had set a 300kB password to test the prehashing implementation, and just continued to use it (via password manager). He was potentially leaking a bit of timing information. I'm not even sure if a password that long made a timing difference that was detectable through network jitter, but if it did and some hypothetical attacker was able to learn that his password was hundreds of kilobytes long, well... good luck. > in fact I use: SHA-512 -> Base64 -> bcrypt when password hashing. Does that make me wrong? No, using a base64 representation of the sha hash makes your scheme perfectly reasonable. Also note that the base64 representation of sha512 is a bit longer than the typical bcrypt input length, so you're shrinking the collision space somewhat by truncating your sha hash. Certainly not enough that it would matter at all, just a point I find interesting. Hypothetically, a binary representation with a pass that removed or replaced NUL bytes would preserve more entropy, but we're well past the point where it matters and would be playing with fire to consider that complexity.

deckard1 5y ago

What's been interesting to me is that during tech interviews, we eagerly have applicants roll their own sorting, or searching, or caching. But you never have questions about crypto or auth. Why one and not the other? Are both not critical to get right? And then I'm reminded of the bug that was hidden in plain sight for 20 years in Programming Pearls[1] in their implementation of a binary search. This same algorithm was copied in Java and elsewhere, near verbatim. So, yeah, maybe don't roll your own searching either. The experts have enough difficulty. Frankly, I don't trust most programmers to get anything right. People toss around "battle-tested" libraries and we all just take it for granted that that's what they are. But I recently found out one incredibly popular package for one incredibly popular language is doing something fairly dumb. Not insecure (that I can tell). Just simply unnecessary. They are signing session cookies. For reasons I could not fathom. Signing doesn't prevent cookie theft. It's just another layer that someone thought would add security over just sending a random string (hopefully your system/language's random generator is actually random). And that's how a lot of these mistakes happen. Let's add sha512 on top of our crypto. Because more = better, right? Speaking of random, there was another popular package out there for generating UUIDs that has a github issue where someone discovered the package was generating collisions. Yikes. [1] https://ai.googleblog.com/2006/06/extra-extra-read-all-about-it-nearly.html https://ai.googleblog.com/2006/06/extra-extra-read-all-about...

yoyonamite 5y ago
Isn't signing the session cookie to prevent cookie tampering? You can send a random string, but you would need to persist the random string somewhere in order to verify against it later
- DougN7 5y ago
  The cookie value has to be stored in some sort of map\dictionary that links to the session data. If the cookie value is a random string, you don’t need to store any more data than you would with a hashed cookie value. Or maybe I’m misunderstanding your point?
  - 5y ago