6 ms·

Is this not the point of a Whitehat bounty program? To entice someone to discover and disclose a bug in a trustworthy manner? If they react this way, and can't

by Qualman 11y ago

Is this not the point of a Whitehat bounty program? To entice someone to discover and disclose a bug in a trustworthy manner? If they react this way, and can't trust people to attempt to find exploitable security holes on their system (even those that yield private keys), then what is the point at all? The only people that find them then, are not going to be as cooperative about it. > Doesn't the author know how long it will take them to recover from this breech? How much it will cost them? This is not the author's fault. He did nothing but disclose bugs that Facebook themselves set in place, and seemed to be very open with them about it, at that.

tptacek 11y ago
No, this is not the point of bug bounties. The point of a bug bounty is to find and fix bugs. That's why they're called "bug bounties". This person took a bug bounty and ran it as a penetration test. Facebook fixed the one bug he found and paid him for it.
- encoderer 11y ago
  What is the protocol for assuming that a bug might have previously been exploited and keys already compromised? Is that just not worried about unless they see evidence in logs?
  - tptacek 11y ago
    I don't know, but that's the security team's job; it is emphatically not the job of a bug bounty researcher to do that.
    - encoderer 11y ago
      I don't know much about this which is why I asked. It seems that severity-based payouts have created incentives that do not match the program rules? Maybe all rce bugs should be paid out on an assumption that if used they'll lead to access to a shell or to user data.
      - tptacek 11y ago
        Severity on a vulnerability assessment is based on the bug itself; it's the severity of the RCE.
        bigiain 11y ago
        Yeah - but it's 100% clear from this that FB wanted to brush the RCE under the carpet with a "not at all severe $2500" classification - without ever admitting to losing their private ssl keys or auth token seeds. He clearly _did_ have a "security vulnerability" that gave him the keys to the kingdom. He knew it, and Facebook know it - and they wanted to pretend it was no big deal. Any bets on how many months till there's a large-scale breach of Facebook user data? The reality of the balance between responsible disclosure and selling an exploit is much easier to evaluate now.
        tptacek 11y ago
        That certainly is the fun and exciting way to read this story.
  - bigiain 11y ago
    Especially considering Alex Stamos apparently requested reassurance that he _hadn't_ accessed particular classes of data - instead of looking in their own presumably non-existent audit logging of people who've had access to the private keys ssl of instgram.com and *.instagram.com!!! (Seriously??? That's some world-class enterprise-grade "moving fast and breaking things"...)
- slewis 11y ago
  Why wouldn't it be considered a bug that accessing one low-permission S3 bucket allowed him to access all the other buckets, including user data and keys?
  - nulltype 11y ago
    It is a bug. But I think the point Facebook is making is that it is impolite to exploit the RCE bug and then access other systems.
    - comex 11y ago
      Both tptacek here and Facebook claim that he found one bug. He found at least two, depending on how you classify things: even if Facebook would not like to admit that their security architecture around token amanagement was/is deficient, and the fuzziness of internal security boundaries makes "bug" somewhat hard to define, it was deficient by industry standards (especially for such a large and tech-focused company), and he got way more access than that RCE should have given him. Whether or not he was supposed to go looking for such additional bug(s), it's discourteous not to at least acknowledge that he found them, and thereby provided Facebook additional value over just finding the RCE.
      - tptacek 11y ago
        If he had told Facebook that at the same time as he reported the credentials he harvested from the database --- which his timeline suggests he could have --- I'd agree with you. But he didn't. He put the credentials in his back pocket so he could pull them out when they suggested he hadn't found his "million dollar bug". And so for a month after they fixed the bug, some fucking rando is walking around with credentials to all of Instagram's AWS assets, totally unbeknownst to anyone at Facebook. They turn down his bid for his "million dollars", and he busts the credentials out on them. You think they're going to thank him? He's lucky it was Stamos and not Mary Ann Davidson.
        lsaferite 11y ago
- Qualman 11y ago
  Bug bounty appears to be a misnomer in this instance. Facebook is specifically asking for reports of security vulnerabilities in their policy: > If you believe you have found a security vulnerability on Facebook, we encourage you to let us know right away.[1] Which then begs the question to me: how do you differentiate an acceptable and unacceptable probing of security vulnerabilities when you can't capture the full impact of an issue without attempting to exploit it to its fullest? Because it is certainly not outlined in their policy. And when you're asking for any whitehat to attempt to discover and disclose security vulnerabilities in your system with only the limpest of guidelines around how to do so, I don't feel that it is warranted to react such as Facebook has here. [1]: https://www.facebook.com/whitehat https://www.facebook.com/whitehat
  - tptacek 11y ago
    I don't know. I feel bad for Alex but if we want to suggest that Facebook's vulnerability disclosure policy was poorly written, I will ruefully agree. When you stand up a bug bounty program, you are giving strangers permission to do something that they would otherwise be prosecuted for doing. You should be extraordinarily careful when you do that, and your rules of engagement should be crystal clear. These weren't.
    - hitekker 11y ago
      EDIT: Having read the CSO's explanation that the guy was using his company work email, it makes more sense why the CSO would contact the company (and explains away the pettiness my comment was referring to) One thing I notice: if the CSO felt like this person did something grossly illegal and irresponsible, why not go straight to the police? Why instead go to the man's employer and speak passively aggressively? Paradoxically, contacting the authorities could have helped facebook's argument. It would communicated to the community at large: "Hey Facebook believes it has clear standing to pursue this guy. Maybe, he really did do something wrong." Instead, what I'm reading is: "Facebook doesn't actually believe what the guy did was illegal per se... but they wanted to spite the guy anyway." For me, it seems petty.
      - 11y ago
- hotgoldminer 11y ago
  I'll rephrase the question. Is the broader vulnerability apparent based on the first discovery OR does it only become clear the further down the rabbit hole you get?
  - tptacek 11y ago
    I don't know. If we're going to speculate, I'll say: the Facebook security team didn't know this system existed (it's a 3rd party admin console on a public IP address!), and their immediate reaction to it was "nuke it from orbit, pay out the bounty for finding it, and forget about it". My guess is that they discovered the AWS credential thing on December 1.
    - flgr 11y ago
      If they discovered the AWS credential thing on December 1 after the security researcher reported it, and wouldn't have discovered it otherwise, and it could be the case that someone else found the exact same attack path first, shouldn't they reward him for making them aware of a problem they would not have otherwise noticed? That they wouldn't have fixed? That others that discovered the same attack path might otherwise still openly exploit to MITM all the traffic, to do arbitrary things with arbitrary user accounts?
    - jessaustin 11y ago
      In your experience, are there other, more careful organizations who would have taken the host offline but saved a disk dump for later investigation?
- blazespin 11y ago
  Which is fine. But threatening to call the cops was really bad.
  - tptacek 11y ago
    You don't know that's what happened, even the researcher didn't say that. You're extrapolating. A much more reasonable and likely explanation of the same set of things we've been told: Alex Stamos called Synack and said that the AWS credentials, which, by the researchers own admission, he'd chosen to retain long after the vulnerability he reported was fixed, had to be deleted, and that if they weren't and the researcher continued to use them, the situation would be out of Stamos' hands and into Facebook legal's, at which point he couldn't keep him from being prosecuted. In that interpretation, Alex isn't threatening the researcher; he's (very reasonably) saying "you cannot use these credentials you've taken from the server, and if you keep doing that, I can't take responsibility for how Facebook will handle this, so you should stop right away before you harm yourself."
    - querulous 11y ago
      it's utterly trivial to revoke and reissue aws access keys. trying to paint this as a necessary security measure is incredibly dishonest. the only plausible reasons to loop in his employer and mention legal remedies are intimidation and incompetence and as you've assured us incompetence is off the table...
    - CRConrad 11y ago
      blazespin > > But threatening to call the cops was really bad. tptacek > You don't know that's what happened, even the researcher didn't say that. You're extrapolating. From Wes' blog (presumably based on his boss' oral description of the call): "Alex then stated that he did not want to have to get Facebook's legal team involved, but that he wasn't sure if this was something he needed to go to law enforcement over." Your bias is apparently badly incapacitating your reading comprehension, because "stated that [...] he wasn't sure if this was something he needed to go to law enforcement over" is exactly threatening to call the cops. Not even your friend mr Stamos, who has presumably read Wes' blog post, is claiming that he didn't. So whom are you saying is lying; Wes, or his boss? Oh, and "(very reasonably) saying 'you cannot use these credentials you've taken from the server, and if you keep doing that, I can't take responsibility for how Facebook will handle this, so you should stop right away before you harm yourself.'" really, really, really sounds like Vito The Baseball Bat "very reasonably" saying "You cannot use this testimony you got off Loanshark Louie, and if you keep doing that, I can't take responsibility for how the boys will handle this, so you should stop right away before you harm yourself." Seeing that as SERIOUSLY (as opposed to sarcastically) "very reasonable"... Well, hello, friendship-bias Bizarro World.
- blazespin 11y ago
  Holy christ that is SO wrong. The system should not be so easy to pivot in that way. That was definitely the real bug. If getting the keys to the kingdom is easy as exploiting a trivial bug than Instagram is really really screwed. As I'm sure it's not the only trivial bug! Instagram should be thanking Wes for the wakeup call instead of making him the enemy.

by Qualman 11y ago

tptacek 11y ago

No, this is not the point of bug bounties. The point of a bug bounty is to find and fix bugs. That's why they're called "bug bounties". This person took a bug bounty and ran it as a penetration test. Facebook fixed the one bug he found and paid him for it.

encoderer 11y ago
What is the protocol for assuming that a bug might have previously been exploited and keys already compromised? Is that just not worried about unless they see evidence in logs?
- tptacek 11y ago
  I don't know, but that's the security team's job; it is emphatically not the job of a bug bounty researcher to do that.
  - encoderer 11y ago
    I don't know much about this which is why I asked. It seems that severity-based payouts have created incentives that do not match the program rules? Maybe all rce bugs should be paid out on an assumption that if used they'll lead to access to a shell or to user data.
    - tptacek 11y ago
      Severity on a vulnerability assessment is based on the bug itself; it's the severity of the RCE.
      - bigiain 11y ago
        Yeah - but it's 100% clear from this that FB wanted to brush the RCE under the carpet with a "not at all severe $2500" classification - without ever admitting to losing their private ssl keys or auth token seeds. He clearly _did_ have a "security vulnerability" that gave him the keys to the kingdom. He knew it, and Facebook know it - and they wanted to pretend it was no big deal. Any bets on how many months till there's a large-scale breach of Facebook user data? The reality of the balance between responsible disclosure and selling an exploit is much easier to evaluate now.
        tptacek 11y ago
        That certainly is the fun and exciting way to read this story.
- bigiain 11y ago
  Especially considering Alex Stamos apparently requested reassurance that he _hadn't_ accessed particular classes of data - instead of looking in their own presumably non-existent audit logging of people who've had access to the private keys ssl of instgram.com and *.instagram.com!!! (Seriously??? That's some world-class enterprise-grade "moving fast and breaking things"...)
slewis 11y ago
Why wouldn't it be considered a bug that accessing one low-permission S3 bucket allowed him to access all the other buckets, including user data and keys?
- nulltype 11y ago
  It is a bug. But I think the point Facebook is making is that it is impolite to exploit the RCE bug and then access other systems.
  - comex 11y ago
    Both tptacek here and Facebook claim that he found one bug. He found at least two, depending on how you classify things: even if Facebook would not like to admit that their security architecture around token amanagement was/is deficient, and the fuzziness of internal security boundaries makes "bug" somewhat hard to define, it was deficient by industry standards (especially for such a large and tech-focused company), and he got way more access than that RCE should have given him. Whether or not he was supposed to go looking for such additional bug(s), it's discourteous not to at least acknowledge that he found them, and thereby provided Facebook additional value over just finding the RCE.
    - tptacek 11y ago
      If he had told Facebook that at the same time as he reported the credentials he harvested from the database --- which his timeline suggests he could have --- I'd agree with you. But he didn't. He put the credentials in his back pocket so he could pull them out when they suggested he hadn't found his "million dollar bug". And so for a month after they fixed the bug, some fucking rando is walking around with credentials to all of Instagram's AWS assets, totally unbeknownst to anyone at Facebook. They turn down his bid for his "million dollars", and he busts the credentials out on them. You think they're going to thank him? He's lucky it was Stamos and not Mary Ann Davidson.
      - lsaferite 11y ago
Qualman 11y ago
Bug bounty appears to be a misnomer in this instance. Facebook is specifically asking for reports of security vulnerabilities in their policy: > If you believe you have found a security vulnerability on Facebook, we encourage you to let us know right away.[1] Which then begs the question to me: how do you differentiate an acceptable and unacceptable probing of security vulnerabilities when you can't capture the full impact of an issue without attempting to exploit it to its fullest? Because it is certainly not outlined in their policy. And when you're asking for any whitehat to attempt to discover and disclose security vulnerabilities in your system with only the limpest of guidelines around how to do so, I don't feel that it is warranted to react such as Facebook has here. [1]: https://www.facebook.com/whitehat https://www.facebook.com/whitehat
- tptacek 11y ago
  I don't know. I feel bad for Alex but if we want to suggest that Facebook's vulnerability disclosure policy was poorly written, I will ruefully agree. When you stand up a bug bounty program, you are giving strangers permission to do something that they would otherwise be prosecuted for doing. You should be extraordinarily careful when you do that, and your rules of engagement should be crystal clear. These weren't.
  - hitekker 11y ago
    EDIT: Having read the CSO's explanation that the guy was using his company work email, it makes more sense why the CSO would contact the company (and explains away the pettiness my comment was referring to) One thing I notice: if the CSO felt like this person did something grossly illegal and irresponsible, why not go straight to the police? Why instead go to the man's employer and speak passively aggressively? Paradoxically, contacting the authorities could have helped facebook's argument. It would communicated to the community at large: "Hey Facebook believes it has clear standing to pursue this guy. Maybe, he really did do something wrong." Instead, what I'm reading is: "Facebook doesn't actually believe what the guy did was illegal per se... but they wanted to spite the guy anyway." For me, it seems petty.
    - 11y ago
hotgoldminer 11y ago
I'll rephrase the question. Is the broader vulnerability apparent based on the first discovery OR does it only become clear the further down the rabbit hole you get?
- tptacek 11y ago
  I don't know. If we're going to speculate, I'll say: the Facebook security team didn't know this system existed (it's a 3rd party admin console on a public IP address!), and their immediate reaction to it was "nuke it from orbit, pay out the bounty for finding it, and forget about it". My guess is that they discovered the AWS credential thing on December 1.
  - flgr 11y ago
    If they discovered the AWS credential thing on December 1 after the security researcher reported it, and wouldn't have discovered it otherwise, and it could be the case that someone else found the exact same attack path first, shouldn't they reward him for making them aware of a problem they would not have otherwise noticed? That they wouldn't have fixed? That others that discovered the same attack path might otherwise still openly exploit to MITM all the traffic, to do arbitrary things with arbitrary user accounts?
  - jessaustin 11y ago
    In your experience, are there other, more careful organizations who would have taken the host offline but saved a disk dump for later investigation?
blazespin 11y ago
Which is fine. But threatening to call the cops was really bad.
- tptacek 11y ago
  You don't know that's what happened, even the researcher didn't say that. You're extrapolating. A much more reasonable and likely explanation of the same set of things we've been told: Alex Stamos called Synack and said that the AWS credentials, which, by the researchers own admission, he'd chosen to retain long after the vulnerability he reported was fixed, had to be deleted, and that if they weren't and the researcher continued to use them, the situation would be out of Stamos' hands and into Facebook legal's, at which point he couldn't keep him from being prosecuted. In that interpretation, Alex isn't threatening the researcher; he's (very reasonably) saying "you cannot use these credentials you've taken from the server, and if you keep doing that, I can't take responsibility for how Facebook will handle this, so you should stop right away before you harm yourself."
  - querulous 11y ago
    it's utterly trivial to revoke and reissue aws access keys. trying to paint this as a necessary security measure is incredibly dishonest. the only plausible reasons to loop in his employer and mention legal remedies are intimidation and incompetence and as you've assured us incompetence is off the table...
  - CRConrad 11y ago
    blazespin > > But threatening to call the cops was really bad. tptacek > You don't know that's what happened, even the researcher didn't say that. You're extrapolating. From Wes' blog (presumably based on his boss' oral description of the call): "Alex then stated that he did not want to have to get Facebook's legal team involved, but that he wasn't sure if this was something he needed to go to law enforcement over." Your bias is apparently badly incapacitating your reading comprehension, because "stated that [...] he wasn't sure if this was something he needed to go to law enforcement over" is exactly threatening to call the cops. Not even your friend mr Stamos, who has presumably read Wes' blog post, is claiming that he didn't. So whom are you saying is lying; Wes, or his boss? Oh, and "(very reasonably) saying 'you cannot use these credentials you've taken from the server, and if you keep doing that, I can't take responsibility for how Facebook will handle this, so you should stop right away before you harm yourself.'" really, really, really sounds like Vito The Baseball Bat "very reasonably" saying "You cannot use this testimony you got off Loanshark Louie, and if you keep doing that, I can't take responsibility for how the boys will handle this, so you should stop right away before you harm yourself." Seeing that as SERIOUSLY (as opposed to sarcastically) "very reasonable"... Well, hello, friendship-bias Bizarro World.
blazespin 11y ago
Holy christ that is SO wrong. The system should not be so easy to pivot in that way. That was definitely the real bug. If getting the keys to the kingdom is easy as exploiting a trivial bug than Instagram is really really screwed. As I'm sure it's not the only trivial bug! Instagram should be thanking Wes for the wakeup call instead of making him the enemy.