4 ms·

I greatly apologize for the mess. Thanks for posting this! We're looking into this right now. - All FB posts via Buffer are temporarily deleted. They will reap

I greatly apologize for the mess. Thanks for posting this! We're looking into this right now. - All FB posts via Buffer are temporarily deleted. They will reappear again once we've enabled the FB app again (which we'll do once we're sure we're not compromised anymore). - All Tweets are also stopped from sending. We'll keep you updated from the @Buffer twitter and FB account. - Leo

FredericJ 13y ago
Do you know what has been compromised? How do you store the Buffer passwords? SHA1(password+salt)?
- mnordhoff_ 13y ago
  Public Service Announcement: "SHA1(password+salt)" is an extremely unsafe way to store passwords. Use PBKDF2, bcrypt or scrypt. Edit (T+7 minutes): Rewritten to not be a jerk.
  - darkarmani 13y ago
    I know that PBKDF1 is deprecated, but using SHA1 is an upgrade to PBKDF1 which was once the standard. I'm not sure if "extremely unsafe" is the right level of worry. You can still do iterations to make the hashing slower, if for some reason you don't do the right thing and use the methods you listed.
  - julien_c 13y ago
    Extremely unsafe? Care to explain why?
    - MasterScrat 13y ago
      Read this: How To Safely Store A Password http://codahale.com/how-to-safely-store-a-password/ http://codahale.com/how-to-safely-store-a-password/
    - nucleardog 13y ago
      Hashes like SHA1 and MD5 are fast. This makes them great for things like verifying file contents... You can hash a lot of data and get your answer very quickly. This is exactly why they are not particularly well suited to passwords. A brute-force attack (since you're salting your passwords I'm sure!) against them isn't a huge undertaking. Just with the equipment in the computer I'm on right now, I'm looking at generating about 11.5b MD5 hashes per second, or 3.1b SHA1 hashes per second. At eight characters a-zA-Z0-9, I'm looking at about 5 and a half minutes to brute force every combination with MD5. Under a day for SHA1. Hashes like bcrypt and scrypt, on the other hand, are designed to be slow. Their complexity factors actually provide means to slow the hashes down even further as hardware becomes faster. Instead of 11.5b/second, you can increase the complexity until you're only able to generate one hash per second... Now it takes you over three and a half centuries to hash what might take one second with MD5. Even ignoring the possibilities of MD5/SHA1 being 'broken', they're simply too fast to be considered for hashing passwords. (Estimates of GPU hashing speed taken from http://golubev.com/gpuest.htm http://golubev.com/gpuest.htm).
      - MarkMc 13y ago
        But it appears that almost all user passwords (99.8%) appear in the top 10,000 list [1]. So even a brute-force attack on a slow hash like bcrypt is pretty cheap in the vast majority of cases. So switching from md5 to bcrypt doesn't improve your security much. [1] http://xato.net/passwords/more-top-worst-passwords/ http://xato.net/passwords/more-top-worst-passwords/
        timtadh 13y ago
        According to that one guy with that one list. I acknowledge that the top N passwords are X% of all user passwords. I sincerely question the 99.8% figure. The problem with doing studies like this is while we have some fairly big password dumps we still do not have the universe. Furthermore, there are some number of un-cracked passwords in the dumps we have. Further complicating the situation are password policies which may reject common passwords. It has been many years since we learned that "password" is the most common password and is commonly disallowed. Therefore 1) it is not useless to not increase your storage security even if Y% of your users use bad passwords as you are protecting 100%-Y% of you users. 2) Y% is probably not 99.8% for you, and if you are worried about it you can take steps to mitigate the problem. ps. He is ignoring punctuation which is an important detail for actually doing the cracking. pps. I appreciate the sentiment (users choose shitty passwords) but not the conclusion (so don't bother storing them well). The proper conclusion is use scrypt/bcrypt and increase the work factor. You can take reasonable steps to protect your users and you should.
  - dylz 13y ago
    @julien_c - broken nearly immediately.
  - sshillo 13y ago
    Is there a good way to 2 way encrypt something in something like rails where the app needs to use the decrypted token. If you have app logic doing decryption then someone can just look at the source to figure out how to do this. Is the only good solution to use a compiled language?
    - mpeg 13y ago
      Compiled programs are as easy to read the source for (at least the relevant bits) as interpreted ones. Still, if you encrypt it and someone gains access only to the database, they wouldn't be able to use the tokens. Every little bit helps.
shortformblog 13y ago
Good work handling this; I'm glad to know that I didn't need to revoke access because you guys were already on top of it.
- pogue 13y ago
  I'd revoke access to the app on Twitter also, and reauthorize it back when its fixed. http://mypermissions.org/ http://mypermissions.org/
  - shortformblog 13y ago
    I did that, but the FB app was already taken care of fortunately.
emhart 13y ago
Best of luck. I, and hopefully everyone else, will be here, ready to buffer more content, when you get it sorted out.
chmars 13y ago
Are you affiliated with Buffer App? Your profile is not that helpful …
- auxbuss 13y ago
  Leo is co-founder of Buffer.
  - chmars 13y ago
    Thanks!
300bps 13y ago
We're looking into this right now. How'd they get into your system? Was it a Buffer Overflow?

FredericJ 13y ago

Do you know what has been compromised? How do you store the Buffer passwords? SHA1(password+salt)?

mnordhoff_ 13y ago
Public Service Announcement: "SHA1(password+salt)" is an extremely unsafe way to store passwords. Use PBKDF2, bcrypt or scrypt. Edit (T+7 minutes): Rewritten to not be a jerk.
- darkarmani 13y ago
  I know that PBKDF1 is deprecated, but using SHA1 is an upgrade to PBKDF1 which was once the standard. I'm not sure if "extremely unsafe" is the right level of worry. You can still do iterations to make the hashing slower, if for some reason you don't do the right thing and use the methods you listed.
- julien_c 13y ago
  Extremely unsafe? Care to explain why?
  - MasterScrat 13y ago
    Read this: How To Safely Store A Password http://codahale.com/how-to-safely-store-a-password/ http://codahale.com/how-to-safely-store-a-password/
  - nucleardog 13y ago
    Hashes like SHA1 and MD5 are fast. This makes them great for things like verifying file contents... You can hash a lot of data and get your answer very quickly. This is exactly why they are not particularly well suited to passwords. A brute-force attack (since you're salting your passwords I'm sure!) against them isn't a huge undertaking. Just with the equipment in the computer I'm on right now, I'm looking at generating about 11.5b MD5 hashes per second, or 3.1b SHA1 hashes per second. At eight characters a-zA-Z0-9, I'm looking at about 5 and a half minutes to brute force every combination with MD5. Under a day for SHA1. Hashes like bcrypt and scrypt, on the other hand, are designed to be slow. Their complexity factors actually provide means to slow the hashes down even further as hardware becomes faster. Instead of 11.5b/second, you can increase the complexity until you're only able to generate one hash per second... Now it takes you over three and a half centuries to hash what might take one second with MD5. Even ignoring the possibilities of MD5/SHA1 being 'broken', they're simply too fast to be considered for hashing passwords. (Estimates of GPU hashing speed taken from http://golubev.com/gpuest.htm http://golubev.com/gpuest.htm).
    - MarkMc 13y ago
      But it appears that almost all user passwords (99.8%) appear in the top 10,000 list [1]. So even a brute-force attack on a slow hash like bcrypt is pretty cheap in the vast majority of cases. So switching from md5 to bcrypt doesn't improve your security much. [1] http://xato.net/passwords/more-top-worst-passwords/ http://xato.net/passwords/more-top-worst-passwords/
      - timtadh 13y ago
        According to that one guy with that one list. I acknowledge that the top N passwords are X% of all user passwords. I sincerely question the 99.8% figure. The problem with doing studies like this is while we have some fairly big password dumps we still do not have the universe. Furthermore, there are some number of un-cracked passwords in the dumps we have. Further complicating the situation are password policies which may reject common passwords. It has been many years since we learned that "password" is the most common password and is commonly disallowed. Therefore 1) it is not useless to not increase your storage security even if Y% of your users use bad passwords as you are protecting 100%-Y% of you users. 2) Y% is probably not 99.8% for you, and if you are worried about it you can take steps to mitigate the problem. ps. He is ignoring punctuation which is an important detail for actually doing the cracking. pps. I appreciate the sentiment (users choose shitty passwords) but not the conclusion (so don't bother storing them well). The proper conclusion is use scrypt/bcrypt and increase the work factor. You can take reasonable steps to protect your users and you should.
- dylz 13y ago
  @julien_c - broken nearly immediately.
- sshillo 13y ago
  Is there a good way to 2 way encrypt something in something like rails where the app needs to use the decrypted token. If you have app logic doing decryption then someone can just look at the source to figure out how to do this. Is the only good solution to use a compiled language?
  - mpeg 13y ago
    Compiled programs are as easy to read the source for (at least the relevant bits) as interpreted ones. Still, if you encrypt it and someone gains access only to the database, they wouldn't be able to use the tokens. Every little bit helps.

shortformblog 13y ago

Good work handling this; I'm glad to know that I didn't need to revoke access because you guys were already on top of it.

pogue 13y ago
I'd revoke access to the app on Twitter also, and reauthorize it back when its fixed. http://mypermissions.org/ http://mypermissions.org/
- shortformblog 13y ago
  I did that, but the FB app was already taken care of fortunately.

emhart 13y ago

Best of luck. I, and hopefully everyone else, will be here, ready to buffer more content, when you get it sorted out.

chmars 13y ago

Are you affiliated with Buffer App? Your profile is not that helpful …

auxbuss 13y ago
Leo is co-founder of Buffer.
- chmars 13y ago
  Thanks!

300bps 13y ago

We're looking into this right now. How'd they get into your system? Was it a Buffer Overflow?