8 ms·

I'm genuinely curious how the decision to use MD5 gets made. Who says, "hey, maybe we should use MD5." And then who responds, "that sounds like a great idea Bob

I'm genuinely curious how the decision to use MD5 gets made. Who says, "hey, maybe we should use MD5." And then who responds, "that sounds like a great idea Bob." Seriously. I've known for years that MD5 is insufficient for hashing passwords and I'm just some random guy. This kind of thing really baffles me.

bpicolo 10y ago
I doubt that decision was made in the last decade. It's surely just something that's been around for a long time and was never upgraded. Still neglectful, but I sincerely doubt it was just a recent engineer's bad decision-making.
stanleydrew 10y ago
Yahoo has been a company for a long time. I imagine your conversation happened round about 1999 when using MD5 wasn't insane. And then they were just slow to upgrade. It's still bad, I'm just saying the conversation about what hash algo to use didn't happen yesterday.
- cm2187 10y ago
  But it's not like if we didn't have a pretty much continuous stream of major data leaks for the past 5 years. Surely yahoo engineers occasionally open a newspaper...
  - Endy 10y ago
    From everything I've read, the engineers did. The problem was that the security team had to go head-to-head with the budget team. And unfortunately, the budget team won - since the upper levels didn't feel that the IT security salaries were a necessary expenditure. And beyond that, there was concern that making people actually change their passwords regularly and requiring anything like security in said passwords was going to discourage users from using Yahoo and send them over to GMail. Unfortunately... that argument wasn't wrong.
    - cm2187 10y ago
      But would it require users to change their password? The way I would have implemented it, but would be keen to know how secure it is, is that you start with the md5 of the password (md5(password)). You then bcrypt or scrypt that md5 (bcrypt(md5(password))) and replace the md5 in your database with the bcrypt hash. When a user logs in, all you need to do is to calculate the md5 first then check that md5 against the bcrypt hash you have stored. I am not a crypto expert but intuitively it doesn't look like I would have weakened the security that way. You can't really attack bcrypt(md5(password)) much more than bcrypt(password). Can you?
      - noonespecial 10y ago
        The problem is in collisions. Md5(password) can yield the same result for many different values of password so simply bcrypting that result means that you start with a restricted possibility space. So less secure. Punts the question to how much less secure. Seems to me it would still be worth it to do and then all new passwords going forward are done correctly.
        cm2187 10y ago
        Agree, but a collision even for md5 is a relatively rare event. When brute-forcing the bcrypt hash, this would reduce the attempts you would need to try against a given hash, but only by a very small factor. With a reasonable work factor, I would assume it would still make a brute force attack impractical at scale. I didn't do the test, but I'd expect that there wouldn't be more than a handful of collisions for the md5 of the 100m most common passwords. [edit] I actually I just did the test on this 10m password list and no collision https://xato.net/today-i-am-releasing-ten-million-passwords-b6278bbe7495#.ekbaa699p https://xato.net/today-i-am-releasing-ten-million-passwords-...
        noonespecial 10y ago
        Very nice. Thanks for that. So yes, this is likely the thing to do in this situation.
- lotyrin 10y ago
  https://en.wikipedia.org/wiki/Salt_(cryptography)#1970s-1980s https://en.wikipedia.org/wiki/Salt_(cryptography)#1970s-1980...
- throwaway34916 10y ago
  I'd like to believe that. However, I was recently asked to test a new website for an organization I volunteer for, and discovered their "forgot password" flow emailed me my plaintext password. I wrote an explanation of why this was bad, and how it could be fixed, to a non-technical friend of mine who works there; he passed my email to the (Bay Area based!) consulting shop that did their website. The shop sent this response: "We do not store passwords as a plain text in database. We have functionality which encrypts and decrypts passwords. We have only ecnrypted passwords in the database. Almost all other servers use one-way encryption. In this case, passwords cannot be decrypted from hashing." Again, this is a Bay Area based shop. For code written in 2016. I was shocked to receive this, but it (among other things) leads me to suspect that there are lot of people out there, in positions of power, who aren't just ignorant, but who actively cling to password-storage anti-patterns. I'm at a loss for how to fix this.
  - crazypyro 10y ago
    Just for clarity, the "forgot password" flow emailed you the current password of the account (not a temporarily one)? That's insane...
    - throwaway34916 10y ago
      Yes, the current password.
  - wtfishackernews 10y ago
    submit the website to http://plaintextoffenders.com/ http://plaintextoffenders.com/
    - syncsynchalt 10y ago
      Ironically, hosted on a Y! site.
x0 10y ago
And nobody ever seemed to say "hey, maybe we should be using something more secure". Yahoo's been around for how many decades, and the fact they were still using MD5 in 2013 is just shameful. Yeah if it was some legacy code from 1993 you can probably excuse it, but I just can't believe after 20 years nobody thought it was a problem. I'm not really a software developer but I really can't imagine it being a huge change. Instead of md5(pass) you could probably just change that to secure_hash(md5(pass), salt), add another column in the database for the salt, and rehash all the passwords. Customers wouldn't notice. Rehashing the databases would take a while, but otherwise that's really not a huge amount of work.
- _asummers 10y ago
  Hashing the hash isn't a good idea, you're reducing the domain of your secure_hash function to the range of md5. The way to do it is to have a "password hash algo version" column and when the user puts in their password, you verify against the hash[algo](password) and rehash with the later version, changing the algo column for that user.
  - aidos 10y ago
    You could do both though. Give much more security in the short term and upgrade anyone else who logged in later. I did ask about the hash of hash thing some time ago and ptacek claimed that's a reasonable thing to do.
  - danielweber 10y ago
    > you're reducing the domain of your secure_hash function to the range of md5. Oh no, only 128 bits. The NSA will be able to brute force one of those passwords in 80 years.
  - nodamage 10y ago
    You need to do both. If you only do the latter, then stale accounts which never log in again will never have their passwords upgraded to the more secure hash. Hashing the hash allows you to replace the md5 hashes immediately, and then you can perform the upgrade if/when the user logs in again.
lomnakkus 10y ago
It gets/got made ~10-15 years ago. (I don't understand the "no salt" thing, though. That was common practice even ~20 years ago on Linux machines, so I'm mildly surprised that it wasn't implemented in this case.)
lucb1e 10y ago
> I'm genuinely curious how the decision to use MD5 gets made. You assume a formal decision was made? I think a manager just went "make them secure" and history was made. That's how it usually seems to happen if it's not a user-facing thing.

bpicolo 10y ago

I doubt that decision was made in the last decade. It's surely just something that's been around for a long time and was never upgraded. Still neglectful, but I sincerely doubt it was just a recent engineer's bad decision-making.

stanleydrew 10y ago

Yahoo has been a company for a long time. I imagine your conversation happened round about 1999 when using MD5 wasn't insane. And then they were just slow to upgrade. It's still bad, I'm just saying the conversation about what hash algo to use didn't happen yesterday.

cm2187 10y ago
But it's not like if we didn't have a pretty much continuous stream of major data leaks for the past 5 years. Surely yahoo engineers occasionally open a newspaper...
- Endy 10y ago
  From everything I've read, the engineers did. The problem was that the security team had to go head-to-head with the budget team. And unfortunately, the budget team won - since the upper levels didn't feel that the IT security salaries were a necessary expenditure. And beyond that, there was concern that making people actually change their passwords regularly and requiring anything like security in said passwords was going to discourage users from using Yahoo and send them over to GMail. Unfortunately... that argument wasn't wrong.
  - cm2187 10y ago
    But would it require users to change their password? The way I would have implemented it, but would be keen to know how secure it is, is that you start with the md5 of the password (md5(password)). You then bcrypt or scrypt that md5 (bcrypt(md5(password))) and replace the md5 in your database with the bcrypt hash. When a user logs in, all you need to do is to calculate the md5 first then check that md5 against the bcrypt hash you have stored. I am not a crypto expert but intuitively it doesn't look like I would have weakened the security that way. You can't really attack bcrypt(md5(password)) much more than bcrypt(password). Can you?
    - noonespecial 10y ago
      The problem is in collisions. Md5(password) can yield the same result for many different values of password so simply bcrypting that result means that you start with a restricted possibility space. So less secure. Punts the question to how much less secure. Seems to me it would still be worth it to do and then all new passwords going forward are done correctly.
      - cm2187 10y ago
        Agree, but a collision even for md5 is a relatively rare event. When brute-forcing the bcrypt hash, this would reduce the attempts you would need to try against a given hash, but only by a very small factor. With a reasonable work factor, I would assume it would still make a brute force attack impractical at scale. I didn't do the test, but I'd expect that there wouldn't be more than a handful of collisions for the md5 of the 100m most common passwords. [edit] I actually I just did the test on this 10m password list and no collision https://xato.net/today-i-am-releasing-ten-million-passwords-b6278bbe7495#.ekbaa699p https://xato.net/today-i-am-releasing-ten-million-passwords-...
        noonespecial 10y ago
        Very nice. Thanks for that. So yes, this is likely the thing to do in this situation.
lotyrin 10y ago
https://en.wikipedia.org/wiki/Salt_(cryptography)#1970s-1980s https://en.wikipedia.org/wiki/Salt_(cryptography)#1970s-1980...
throwaway34916 10y ago
I'd like to believe that. However, I was recently asked to test a new website for an organization I volunteer for, and discovered their "forgot password" flow emailed me my plaintext password. I wrote an explanation of why this was bad, and how it could be fixed, to a non-technical friend of mine who works there; he passed my email to the (Bay Area based!) consulting shop that did their website. The shop sent this response: "We do not store passwords as a plain text in database. We have functionality which encrypts and decrypts passwords. We have only ecnrypted passwords in the database. Almost all other servers use one-way encryption. In this case, passwords cannot be decrypted from hashing." Again, this is a Bay Area based shop. For code written in 2016. I was shocked to receive this, but it (among other things) leads me to suspect that there are lot of people out there, in positions of power, who aren't just ignorant, but who actively cling to password-storage anti-patterns. I'm at a loss for how to fix this.
- crazypyro 10y ago
  Just for clarity, the "forgot password" flow emailed you the current password of the account (not a temporarily one)? That's insane...
  - throwaway34916 10y ago
    Yes, the current password.
- wtfishackernews 10y ago
  submit the website to http://plaintextoffenders.com/ http://plaintextoffenders.com/
  - syncsynchalt 10y ago
    Ironically, hosted on a Y! site.

x0 10y ago

And nobody ever seemed to say "hey, maybe we should be using something more secure". Yahoo's been around for how many decades, and the fact they were still using MD5 in 2013 is just shameful. Yeah if it was some legacy code from 1993 you can probably excuse it, but I just can't believe after 20 years nobody thought it was a problem. I'm not really a software developer but I really can't imagine it being a huge change. Instead of md5(pass) you could probably just change that to secure_hash(md5(pass), salt), add another column in the database for the salt, and rehash all the passwords. Customers wouldn't notice. Rehashing the databases would take a while, but otherwise that's really not a huge amount of work.

_asummers 10y ago
Hashing the hash isn't a good idea, you're reducing the domain of your secure_hash function to the range of md5. The way to do it is to have a "password hash algo version" column and when the user puts in their password, you verify against the hash[algo](password) and rehash with the later version, changing the algo column for that user.
- aidos 10y ago
  You could do both though. Give much more security in the short term and upgrade anyone else who logged in later. I did ask about the hash of hash thing some time ago and ptacek claimed that's a reasonable thing to do.
- danielweber 10y ago
  > you're reducing the domain of your secure_hash function to the range of md5. Oh no, only 128 bits. The NSA will be able to brute force one of those passwords in 80 years.
- nodamage 10y ago
  You need to do both. If you only do the latter, then stale accounts which never log in again will never have their passwords upgraded to the more secure hash. Hashing the hash allows you to replace the md5 hashes immediately, and then you can perform the upgrade if/when the user logs in again.

lomnakkus 10y ago

It gets/got made ~10-15 years ago. (I don't understand the "no salt" thing, though. That was common practice even ~20 years ago on Linux machines, so I'm mildly surprised that it wasn't implemented in this case.)

lucb1e 10y ago

> I'm genuinely curious how the decision to use MD5 gets made. You assume a formal decision was made? I think a manager just went "make them secure" and history was made. That's how it usually seems to happen if it's not a user-facing thing.