4 ms·

The "bug" here is that they aren't really keeping track of their AWS buckets and keys at all. Least privilege, access logging, remote IP flagging, etc. These op

The "bug" here is that they aren't really keeping track of their AWS buckets and keys at all. Least privilege, access logging, remote IP flagging, etc. These operational failures are ostensibly the responsibility of the CSO. I'm not saying this researcher was 100% in the right, but this is the CSO ass covering. "Don't pay attention to the obvious operational deficits, the problem is the researcher overreaching." A simple phone call directly to the researcher that cut through the bullshit would have made everything better. But he had to make sure it didn't get out and the only way he could do that was by using the only leverage he had: The researcher's employer.

tptacek 11y ago
Alex has in the last few months built one of the best teams in application security at Facebook (Facebook security is now seemingly most of O.G. iSEC Partners). I get it, everyone hates big companies and especially Facebook evil Facebook but, come on. They know what they're doing. If you understand how security works inside of big companies, this is a really silly theory to run with. CSOs are happy when shit like this gets discovered, because it gives them ammunition to get the rest of the company to adjust policies. If you were working from the understanding that a CSO comes in and just immediately tells a team of (what is it) NINE THOUSAND developers how to do stuff differently... no. That's not how it works. The problem is that nobody at Facebook with the possible exception of like 10 people none of whom are Alex can make huge operational changes like "change all the ways we store keys across an entire huge business unit". So, you tell Alex you took AWS credentials he didn't know existed and you're going to start mining them for a story you're bringing to the media, and now Alex is in a position where he's NOT ALLOWED to sit back and try to manage the situation himself. Delete the keys or I have to tell legal what's happening. The researcher NEEDED TO HEAR THAT.
- brazzledazzle 11y ago
  I don't doubt that he's put together a great application security team. Or that he even knows his shit. And I do understand how it works. CSOs are happy when this kind of shit gets discovered when they can't get other teams onboard to fix it. They're unhappy when it gets discovered when they intentionally ignore it in favor of another initiative (particularly if there's a paper trail showing that someone brought it to their attention). Or when they've already spent a bunch of money and resources fixing it only for everyone to find that they haven't fixed it at all. There are basic things you can do to mitigate or isolate damage in AWS and they either aren't doing it or have done it badly. Even if he couldn't convince the rest of the company that god-mode keys are bad, he still could have built out some basic infrastructure to track when and where they keys were being used from so red flags could be raised when some random IP address is being used to pull down several buckets.
- ju-st 11y ago
  There are enough laws against "cybercrime". If Alex felt threatened he should have escalated the issue to the FBI. There is no single reason to call the employer. By doing so Alex has threatened Wesley to fuck up his life. edit: Or -after calling the CEO- he should have contacted Wesley directly and so they could deescalate the problem together.
- Retric 11y ago
  Relative security teams are almost useless. In 2-3 years FB might have it's shit together, but three months is no where near long enough to fix there problems.
  - blazespin 11y ago
    Agreed, after looking at this linked in Profile, it's hard to blame Alex for the problems as he's only been there a short while. However, he can be blamed for creating all this unnecessary drama.
- slantyyz 11y ago
  >> Delete the keys or I have to tell legal what's happening. >> The researcher NEEDED TO HEAR THAT. I'm not in security, but from the outside looking in, how things worked out just doesn't smell right. If "the researcher NEEDED TO HEAR THAT" is the priority, then why waste time looking up who the guy works for and calling them instead? The simplest and most obvious way to tell the researcher is to tell him directly in the clearest way possible. It isn't as though there wasn't a pre-existing line of communication with the researcher.
  - sangnoir 11y ago
    My reading of tptacek's subtext is that Facebook wanted to show the researcher that they were really, ALL-CAPS serious, as in "get you fired and ruin-your-livelihood if you don't stop" serious. These mafia tactics are fine because the Facebook CSO "built a good team and knows what he is doing"
    - CRConrad 11y ago
      ...which all adds up to a smell of "tptacek knows that team is good, because he's on it".
    - slantyyz 11y ago
      If FB wanted to show the researcher that they were really, ALL-CAPS serious, then they would talk to him directly as in "You've got stolen data and we're going have the FBI arrest you, seize your computers and put you in jail ruin-your-livelihood if you don't stop" serious. So I still don't see how calling the guy's boss trumps that in terms of scariness. Because if I'm the wronged party (i.e., FB), that's what I'd do if I couldn't resolve it amicably.
      - sangnoir 11y ago
        If we are disagreeing, I don't quite follow your argument - I never said that this was the worst/scariest thing Facebook could to do (there's no upper limit). What I meant was that the action by Facebook was intended to intimidate (and not that the specific form of intimidation was the worst possible)
- pdeuchler 11y ago
  You're perfectly right, but his employer didn't need to hear it. And that's the whole crux of the matter.
  - yeukhon 11y ago
    If you read the article, his company does security research and found a vulnerability in Hotmail. Plus he was using his company's email address. > At this point, it was reasonable to believe that Wes was operating on behalf of Synack. His account on our portal mentions Synack as his affiliation, he has interacted with us using a synack.com email address, and he has written blog posts that are used by Synack for marketing purposes. That's a big mistake. DO NOT EVER USE YOUR COMPANY EMAIL ADDRESS if you are doing this on your own. The employer has the right to know. Imagine using a company email address on Ashley.com. Yeah, plenty of people were embarrassed after that hack.
    - hectormalot 11y ago
      Actually his write up makes pretty clear that he didn't use his company email until after Alex went over his head to the CEO. Second, everything else being equal, Alex going to the CEO without calling or mailing the researcher first was a mistake. Going to someone's boss and saying "please do something, I don't want to get the lawyers involved" IS an implicit legal threat, both to synack and the researcher.
      - tptacek 11y ago
        I like how we're talking about Stamos warning a guy running around with stolen AWS credentials for all of Instagram in the same fashion as we'd talk about a DMCA threat. "Implicit legal threat"? There's nothing "implicit" or subtle about what was happening here.
        yarou 11y ago
        Then why the immediate escalation? Wouldn't it have made more sense to contact the researcher directly, rather than using his position of power to pressure the researcher's company's CEO? Why not assume good faith? (Which is what I would think a white hat bug bounty program should assume)
- girvo 11y ago
  > The researcher NEEDED TO HEAR THAT. I don't disagree. But why go through his employer, when they already had a direct line to the researcher himself?
  - sangnoir 11y ago
    Intimidation.
- blazespin 11y ago
  If you understood how big companies work, you'd know it takes more than a few months to build "one of the best teams". This is one thing in Alex's favor though, he's new to the job. Still, if you also understood how big companies work, you'd know that everyone hates the drama queen. The right move here would have not been to threaten Wes, pay him, and just update the policy. Lesson learned for Alex and his friends: Do not threaten individual contributors or suffer massive freaking drama. Thank you internet.
- sandaru1 11y ago
  In any case, there is one question remains. How do facebook defines a "million dollar" bug if the security team is not aware of the damage it can do. Since this is not the first time this bug was reported, did they actually give a big bounty to the first person who did the initial report(Given that it can lead to this much damage)? Or just another small bounty saying that it's not a very important security flaw.
balls187 11y ago
> I'm not saying this researcher was 100% in the right, but this is the CSO ass covering. "Don't pay attention to the obvious operational deficits, the problem is the researcher overreaching." The response from FB's CSO is very specific to a very specific blog publication. Not regarding the flaws in how their AWS Buckets are used.
- brazzledazzle 11y ago
  I'm not sure what you're getting at.
  - balls187 11y ago
    Your statement: > "Don't pay attention to the obvious operational deficits, the problem is the researcher overreaching." mischaracterizes what the response by FB CSO as one that is attempting to draw criticism away from operation flaws by instead placing focus/blame on the researchers methodology.
    - timv 11y ago
      I disagree. A security researcher went public with a story of "I found this massive security hole and Facebook tried to avoid paying what I thought it was worth, and then threatened me with legal action" The response that Alex thinks he needs to make is "my actions were reasonable because ..." From external appearances it seems as though he is more concerned about looking like a heavy-handed, lawyer-invoking, CSO than the publicity around FB having an unpatched RCE that allowed access to highly-privileged AWS keys. What he chooses to write about is reflection of what he saw as the most important news in the original blog post. I suspect he's actually right. The blog post will probably raise more bad publicity around the way FB handled the research & disclosure than the existence of the bug, and it's the piece that needs to be resolved well.
    - brazzledazzle 11y ago
      You're right, that was the purpose of trying to keep him quiet by contacting the CE-freaking-O of his place of employment with an implicit legal threat. The blog post is an attempt to do damage control when he realized the researcher wasn't going to put up with that and went public.

tptacek 11y ago

Alex has in the last few months built one of the best teams in application security at Facebook (Facebook security is now seemingly most of O.G. iSEC Partners). I get it, everyone hates big companies and especially Facebook evil Facebook but, come on. They know what they're doing. If you understand how security works inside of big companies, this is a really silly theory to run with. CSOs are happy when shit like this gets discovered, because it gives them ammunition to get the rest of the company to adjust policies. If you were working from the understanding that a CSO comes in and just immediately tells a team of (what is it) NINE THOUSAND developers how to do stuff differently... no. That's not how it works. The problem is that nobody at Facebook with the possible exception of like 10 people none of whom are Alex can make huge operational changes like "change all the ways we store keys across an entire huge business unit". So, you tell Alex you took AWS credentials he didn't know existed and you're going to start mining them for a story you're bringing to the media, and now Alex is in a position where he's NOT ALLOWED to sit back and try to manage the situation himself. Delete the keys or I have to tell legal what's happening. The researcher NEEDED TO HEAR THAT.

brazzledazzle 11y ago
I don't doubt that he's put together a great application security team. Or that he even knows his shit. And I do understand how it works. CSOs are happy when this kind of shit gets discovered when they can't get other teams onboard to fix it. They're unhappy when it gets discovered when they intentionally ignore it in favor of another initiative (particularly if there's a paper trail showing that someone brought it to their attention). Or when they've already spent a bunch of money and resources fixing it only for everyone to find that they haven't fixed it at all. There are basic things you can do to mitigate or isolate damage in AWS and they either aren't doing it or have done it badly. Even if he couldn't convince the rest of the company that god-mode keys are bad, he still could have built out some basic infrastructure to track when and where they keys were being used from so red flags could be raised when some random IP address is being used to pull down several buckets.
ju-st 11y ago
There are enough laws against "cybercrime". If Alex felt threatened he should have escalated the issue to the FBI. There is no single reason to call the employer. By doing so Alex has threatened Wesley to fuck up his life. edit: Or -after calling the CEO- he should have contacted Wesley directly and so they could deescalate the problem together.
Retric 11y ago
Relative security teams are almost useless. In 2-3 years FB might have it's shit together, but three months is no where near long enough to fix there problems.
- blazespin 11y ago
  Agreed, after looking at this linked in Profile, it's hard to blame Alex for the problems as he's only been there a short while. However, he can be blamed for creating all this unnecessary drama.
slantyyz 11y ago
>> Delete the keys or I have to tell legal what's happening. >> The researcher NEEDED TO HEAR THAT. I'm not in security, but from the outside looking in, how things worked out just doesn't smell right. If "the researcher NEEDED TO HEAR THAT" is the priority, then why waste time looking up who the guy works for and calling them instead? The simplest and most obvious way to tell the researcher is to tell him directly in the clearest way possible. It isn't as though there wasn't a pre-existing line of communication with the researcher.
- sangnoir 11y ago
  My reading of tptacek's subtext is that Facebook wanted to show the researcher that they were really, ALL-CAPS serious, as in "get you fired and ruin-your-livelihood if you don't stop" serious. These mafia tactics are fine because the Facebook CSO "built a good team and knows what he is doing"
  - CRConrad 11y ago
    ...which all adds up to a smell of "tptacek knows that team is good, because he's on it".
  - slantyyz 11y ago
    If FB wanted to show the researcher that they were really, ALL-CAPS serious, then they would talk to him directly as in "You've got stolen data and we're going have the FBI arrest you, seize your computers and put you in jail ruin-your-livelihood if you don't stop" serious. So I still don't see how calling the guy's boss trumps that in terms of scariness. Because if I'm the wronged party (i.e., FB), that's what I'd do if I couldn't resolve it amicably.
    - sangnoir 11y ago
      If we are disagreeing, I don't quite follow your argument - I never said that this was the worst/scariest thing Facebook could to do (there's no upper limit). What I meant was that the action by Facebook was intended to intimidate (and not that the specific form of intimidation was the worst possible)
pdeuchler 11y ago
You're perfectly right, but his employer didn't need to hear it. And that's the whole crux of the matter.
- yeukhon 11y ago
  If you read the article, his company does security research and found a vulnerability in Hotmail. Plus he was using his company's email address. > At this point, it was reasonable to believe that Wes was operating on behalf of Synack. His account on our portal mentions Synack as his affiliation, he has interacted with us using a synack.com email address, and he has written blog posts that are used by Synack for marketing purposes. That's a big mistake. DO NOT EVER USE YOUR COMPANY EMAIL ADDRESS if you are doing this on your own. The employer has the right to know. Imagine using a company email address on Ashley.com. Yeah, plenty of people were embarrassed after that hack.
  - hectormalot 11y ago
    Actually his write up makes pretty clear that he didn't use his company email until after Alex went over his head to the CEO. Second, everything else being equal, Alex going to the CEO without calling or mailing the researcher first was a mistake. Going to someone's boss and saying "please do something, I don't want to get the lawyers involved" IS an implicit legal threat, both to synack and the researcher.
    - tptacek 11y ago
      I like how we're talking about Stamos warning a guy running around with stolen AWS credentials for all of Instagram in the same fashion as we'd talk about a DMCA threat. "Implicit legal threat"? There's nothing "implicit" or subtle about what was happening here.
      - yarou 11y ago
        Then why the immediate escalation? Wouldn't it have made more sense to contact the researcher directly, rather than using his position of power to pressure the researcher's company's CEO? Why not assume good faith? (Which is what I would think a white hat bug bounty program should assume)
girvo 11y ago
> The researcher NEEDED TO HEAR THAT. I don't disagree. But why go through his employer, when they already had a direct line to the researcher himself?
- sangnoir 11y ago
  Intimidation.
blazespin 11y ago
If you understood how big companies work, you'd know it takes more than a few months to build "one of the best teams". This is one thing in Alex's favor though, he's new to the job. Still, if you also understood how big companies work, you'd know that everyone hates the drama queen. The right move here would have not been to threaten Wes, pay him, and just update the policy. Lesson learned for Alex and his friends: Do not threaten individual contributors or suffer massive freaking drama. Thank you internet.
sandaru1 11y ago
In any case, there is one question remains. How do facebook defines a "million dollar" bug if the security team is not aware of the damage it can do. Since this is not the first time this bug was reported, did they actually give a big bounty to the first person who did the initial report(Given that it can lead to this much damage)? Or just another small bounty saying that it's not a very important security flaw.

balls187 11y ago

> I'm not saying this researcher was 100% in the right, but this is the CSO ass covering. "Don't pay attention to the obvious operational deficits, the problem is the researcher overreaching." The response from FB's CSO is very specific to a very specific blog publication. Not regarding the flaws in how their AWS Buckets are used.

brazzledazzle 11y ago
I'm not sure what you're getting at.
- balls187 11y ago
  Your statement: > "Don't pay attention to the obvious operational deficits, the problem is the researcher overreaching." mischaracterizes what the response by FB CSO as one that is attempting to draw criticism away from operation flaws by instead placing focus/blame on the researchers methodology.
  - timv 11y ago
    I disagree. A security researcher went public with a story of "I found this massive security hole and Facebook tried to avoid paying what I thought it was worth, and then threatened me with legal action" The response that Alex thinks he needs to make is "my actions were reasonable because ..." From external appearances it seems as though he is more concerned about looking like a heavy-handed, lawyer-invoking, CSO than the publicity around FB having an unpatched RCE that allowed access to highly-privileged AWS keys. What he chooses to write about is reflection of what he saw as the most important news in the original blog post. I suspect he's actually right. The blog post will probably raise more bad publicity around the way FB handled the research & disclosure than the existence of the bug, and it's the piece that needs to be resolved well.
  - brazzledazzle 11y ago
    You're right, that was the purpose of trying to keep him quiet by contacting the CE-freaking-O of his place of employment with an implicit legal threat. The blog post is an attempt to do damage control when he realized the researcher wasn't going to put up with that and went public.