6 ms·

A blameless post-mortem of USA vs. Joseph Sullivan

66 points by mik3y 4y ago

deleted 4y ago
[deleted]
robertlagrant 4y ago
> Takeaways Missed opportunity to call this section Uber Eats.
mcguire 4y ago
The US Attorney's Office press release about the case: https://www.justice.gov/usao-ndca/pr/former-chief-security-officer-uber-convicted-federal-charges-covering-data-breach https://www.justice.gov/usao-ndca/pr/former-chief-security-o...
joecool1029 4y ago
courtlistener page with the docket for the case: https://www.courtlistener.com/docket/18443231/united-states-v-sullivan/ https://www.courtlistener.com/docket/18443231/united-states-... (for the the case docket if some of HN wants to use recap extension https://free.law/recap https://free.law/recap and burn some PACER credit. It's free to make a PACER account and use up to $30 a quarter, they won't bill you).
dmix 4y ago
> The breach remained undetected for an unknown period of months before an interview with an engineer at a competing company disclosed that an executive at their employer had a copy of an Uber database What’s the backstory here? Did an Uber competitor buy the database from a hacker? Then Uber found out which is how they found the data breach happened? Am I reading that right? That sounds very shady whoever the competitor was.
- mikeyouse 4y ago
  It was Lyft; https://www.reuters.com/article/uk-uber-tech-lyft-hacking-exclusive/exclusive-in-lawsuit-over-hacking-uber-probes-ip-address-assigned-to-lyft-exec-sources-idUKKCN0S20D020151008 https://www.reuters.com/article/uk-uber-tech-lyft-hacking-ex... Reading between some lines, what Uber thinks happened is that Chris Lambert, the CTO of Lyft accessed the Github repository that had the improperly stored key (they know the IP address of the only person who they couldn't rule out and that IP was associated with Chris elsewhere online). But then the actual hack using those credentials was carried out via NordVPN so they can't be sure who actually downloaded the material. Later on, Uber was interviewing a Lyft engineer who let them know that Lyft had a copy of the driver database, so they reverse engineered how they could have downloaded it and found their exposed credentials.
  - 0xBAADA555 4y ago
    I'm genuinely surprised because I would have assumed NordVPN keeps user activity logs regardless of what they say (what all those services say, really).
    - HPsquared 4y ago
      If they did, they probably wouldn't burn that resource over some petty civil matter.
      - public_defender 4y ago
        1) It was a petty criminal matter. 2) All the advertisement bluster in the world melts away at the arrival of the first subpoena, search warrant, or national security letter. I'm not saying that there must be no NordVPN logs, but I am saying that if they existed and they were subpoenaed, it would be really, really surprising if Nord didn't immediately turn them over.
mannykannot 4y ago
It is helpful that this was written from a blameless perspective, as it remains clear that the attempt to retroactively re-designate the breach as an authorized bug-bounty act was deceptive and self-serving.
happyopossum 4y ago
> My analysis will treat this as an accident. How does that work for a scenario where specific and intentional actions were taken by an individual or group of individuals, that may or may not have been illegal? You can’t analyze an armed robbery prosecution as an accident without ignoring all of the most significant aspects of the case.
- projektfu 4y ago
  In fairness, this is a conviction for failing to call the police (misprision) when your boss decided to cover up a data breach, not for committing armed robbery. Obstruction of justice as well. On another note, future whistleblowers could, I suppose, cite this as a reason why they cannot keep their mouth shut.
- tptacek 4y ago
  The point of the postmortem is to educate people as to what to do differently. You get more value out of the "accident" frame than out of the "malice" frame, which has already been intensively explored by the DOJ. Most organizations aren't going to be in a position of having a formal federal prosecutor serving as CSO and Deputy General Counsel making grave errors in fitting a report to a bug bounty process instead of a criminal investigation, but it's easy to imagine a variety of organizations landing in a variety of different scrapes about how to route reports that might trigger breach report liability.
- atoav 4y ago
  > You can’t analyze an armed robbery prosecution as an accident without ignoring all of the most significant aspects of the case. Depends on the scope of the analysis. You can analyze why the security systems allowed for that armed robbery to happen and recommend how to prevent this from happening in the future. You can analyze what societal factors and incentive structures lead to this and similar robberies and how to reduce the likelyhood of that occuring in the future. For all of this the question of guilt doesn't matter at all.
- ineptech 4y ago
  The purpose of a retro is to identify a process that will keep some bad thing from happening again; a process that keeps the bad thing from happening for any reason is strictly better than a process that keeps the bad thing from happening due to malice. If your infra engineers can't expose your PII on accident, they can't expose it on purpose either. To extend your analogy, it's the difference between locking your house (which protects your valuables from thieves, but not your garden hose and wheelbarrow from your drunken neighbor) vs. locking your shed as well.
IfOnlyYouKnew 4y ago
So… a federal jury found this guy guilty, but here we have a friend of his who is going to be totally neutral in a reevaluation? So they set out to describe it as „an accident“ because „blameless post-mortems“ are something people really like? Also this article falls into the trap of trying to sound smart by using, sorry, „by effecting the usage of“ big fancy words. I’ve read Supreme Court transcripts and judgements, and I can understand them. This is overtaxing my buzzword ingestion.
- floober 4y ago
  > So they set out to describe it as „an accident“ because „blameless post-mortems“ are something people really like? As someone who has operated bug bounty programs, understanding what processes might have prevented things from going off the rails _in spite of_ internal actors with different motivations is very helpful to me. Placing all of the blame on an individual removes the opportunity to improve things.
  - simoncion 4y ago
    > Placing all of the blame on an individual removes the opportunity to improve things. It seems to me that there's another option. Describe the problem thusly: > A Lyft employee grabbed our data storage access keys from Github. He, or someone else then used these keys to grab PII that Uber was legally required to safeguard. Uber management and/or legal actively worked to cover all of this up and mislead the FTC about the nature and size of the breach. > > Given these facts, what processes and procedures can we change or create to ensure that the PII we're charged with safeguarding remains safe and guarded, that any threat to or breach of said information is detected as soon as is reasonably possible, and that any attempts of management and/or legal to cover up any such incidents are detected and reported to the appropriate authorities?
  - stefan_ 4y ago
    But the bug bounty policy was very clear on all of this and this extortionist never concealed his intentions. And all this text can come up with is "what if we loop in even more people". Indeed this description made it very clear that the existing processes were intentionally subverted; what can more processes do for avoiding that when it happens by decision of the CSO and CEO?
tptacek 4y ago
In case you're unfamiliar with him: if you're a startup person, Ryan McGeehan (the author here) has some of the best resources on building security programs available on the Internet: https://scrty.io/ https://scrty.io/
- encryptluks2 4y ago
  I can't fathom why they would then use Medium to publish an article like this instead of their own website.
gsk22 4y ago
> I don’t believe Joe is a criminal, but my personal opinions about his guilt don’t matter. Then why bring it up? > My analysis will treat this as an accident. Isn't the point of an analysis to determine what happened? Not to start with a preconceived idea and make the analysis fit that? It's hard to trust a source repeatedly claiming to be neutral after spending the first few paragraphs espousing their biases.
- jasonhansel 4y ago
  Usually, in a blameless analysis, you avoid overtly criticizing anyone or assigning blame, but you at least say what different people did. The OP doesn't seem to go into much detail about the exact actions Sullivan took, which seems like a major omission.
  - tptacek 4y ago
    It's not about his actions, it's about the processes that enabled them.
    - deleted 4y ago
      [deleted]
- deleted 4y ago
  [deleted]
your_username 4y ago
jasonhansel 4y ago
My takeaway: whenever private data has inadvertently become available to outside actors, you should treat as a breach, even if you don't have evidence of exfiltration or malicious use, and even if you first found out about the issue from a bug bounty program (legitimate or otherwise).

66 points by mik3y 4y ago

deleted 4y ago

[deleted]

robertlagrant 4y ago

> Takeaways Missed opportunity to call this section Uber Eats.

mcguire 4y ago

The US Attorney's Office press release about the case: https://www.justice.gov/usao-ndca/pr/former-chief-security-officer-uber-convicted-federal-charges-covering-data-breach https://www.justice.gov/usao-ndca/pr/former-chief-security-o...

joecool1029 4y ago

courtlistener page with the docket for the case: https://www.courtlistener.com/docket/18443231/united-states-v-sullivan/ https://www.courtlistener.com/docket/18443231/united-states-... (for the the case docket if some of HN wants to use recap extension https://free.law/recap https://free.law/recap and burn some PACER credit. It's free to make a PACER account and use up to $30 a quarter, they won't bill you).

dmix 4y ago

> The breach remained undetected for an unknown period of months before an interview with an engineer at a competing company disclosed that an executive at their employer had a copy of an Uber database What’s the backstory here? Did an Uber competitor buy the database from a hacker? Then Uber found out which is how they found the data breach happened? Am I reading that right? That sounds very shady whoever the competitor was.

mikeyouse 4y ago
It was Lyft; https://www.reuters.com/article/uk-uber-tech-lyft-hacking-exclusive/exclusive-in-lawsuit-over-hacking-uber-probes-ip-address-assigned-to-lyft-exec-sources-idUKKCN0S20D020151008 https://www.reuters.com/article/uk-uber-tech-lyft-hacking-ex... Reading between some lines, what Uber thinks happened is that Chris Lambert, the CTO of Lyft accessed the Github repository that had the improperly stored key (they know the IP address of the only person who they couldn't rule out and that IP was associated with Chris elsewhere online). But then the actual hack using those credentials was carried out via NordVPN so they can't be sure who actually downloaded the material. Later on, Uber was interviewing a Lyft engineer who let them know that Lyft had a copy of the driver database, so they reverse engineered how they could have downloaded it and found their exposed credentials.
- 0xBAADA555 4y ago
  I'm genuinely surprised because I would have assumed NordVPN keeps user activity logs regardless of what they say (what all those services say, really).
  - HPsquared 4y ago
    If they did, they probably wouldn't burn that resource over some petty civil matter.
    - public_defender 4y ago
      1) It was a petty criminal matter. 2) All the advertisement bluster in the world melts away at the arrival of the first subpoena, search warrant, or national security letter. I'm not saying that there must be no NordVPN logs, but I am saying that if they existed and they were subpoenaed, it would be really, really surprising if Nord didn't immediately turn them over.

mannykannot 4y ago

It is helpful that this was written from a blameless perspective, as it remains clear that the attempt to retroactively re-designate the breach as an authorized bug-bounty act was deceptive and self-serving.

happyopossum 4y ago

> My analysis will treat this as an accident. How does that work for a scenario where specific and intentional actions were taken by an individual or group of individuals, that may or may not have been illegal? You can’t analyze an armed robbery prosecution as an accident without ignoring all of the most significant aspects of the case.

projektfu 4y ago
In fairness, this is a conviction for failing to call the police (misprision) when your boss decided to cover up a data breach, not for committing armed robbery. Obstruction of justice as well. On another note, future whistleblowers could, I suppose, cite this as a reason why they cannot keep their mouth shut.
tptacek 4y ago
The point of the postmortem is to educate people as to what to do differently. You get more value out of the "accident" frame than out of the "malice" frame, which has already been intensively explored by the DOJ. Most organizations aren't going to be in a position of having a formal federal prosecutor serving as CSO and Deputy General Counsel making grave errors in fitting a report to a bug bounty process instead of a criminal investigation, but it's easy to imagine a variety of organizations landing in a variety of different scrapes about how to route reports that might trigger breach report liability.
atoav 4y ago
> You can’t analyze an armed robbery prosecution as an accident without ignoring all of the most significant aspects of the case. Depends on the scope of the analysis. You can analyze why the security systems allowed for that armed robbery to happen and recommend how to prevent this from happening in the future. You can analyze what societal factors and incentive structures lead to this and similar robberies and how to reduce the likelyhood of that occuring in the future. For all of this the question of guilt doesn't matter at all.
ineptech 4y ago
The purpose of a retro is to identify a process that will keep some bad thing from happening again; a process that keeps the bad thing from happening for any reason is strictly better than a process that keeps the bad thing from happening due to malice. If your infra engineers can't expose your PII on accident, they can't expose it on purpose either. To extend your analogy, it's the difference between locking your house (which protects your valuables from thieves, but not your garden hose and wheelbarrow from your drunken neighbor) vs. locking your shed as well.

IfOnlyYouKnew 4y ago

So… a federal jury found this guy guilty, but here we have a friend of his who is going to be totally neutral in a reevaluation? So they set out to describe it as „an accident“ because „blameless post-mortems“ are something people really like? Also this article falls into the trap of trying to sound smart by using, sorry, „by effecting the usage of“ big fancy words. I’ve read Supreme Court transcripts and judgements, and I can understand them. This is overtaxing my buzzword ingestion.

floober 4y ago
> So they set out to describe it as „an accident“ because „blameless post-mortems“ are something people really like? As someone who has operated bug bounty programs, understanding what processes might have prevented things from going off the rails _in spite of_ internal actors with different motivations is very helpful to me. Placing all of the blame on an individual removes the opportunity to improve things.
- simoncion 4y ago
  > Placing all of the blame on an individual removes the opportunity to improve things. It seems to me that there's another option. Describe the problem thusly: > A Lyft employee grabbed our data storage access keys from Github. He, or someone else then used these keys to grab PII that Uber was legally required to safeguard. Uber management and/or legal actively worked to cover all of this up and mislead the FTC about the nature and size of the breach. > > Given these facts, what processes and procedures can we change or create to ensure that the PII we're charged with safeguarding remains safe and guarded, that any threat to or breach of said information is detected as soon as is reasonably possible, and that any attempts of management and/or legal to cover up any such incidents are detected and reported to the appropriate authorities?
- stefan_ 4y ago
  But the bug bounty policy was very clear on all of this and this extortionist never concealed his intentions. And all this text can come up with is "what if we loop in even more people". Indeed this description made it very clear that the existing processes were intentionally subverted; what can more processes do for avoiding that when it happens by decision of the CSO and CEO?

tptacek 4y ago

In case you're unfamiliar with him: if you're a startup person, Ryan McGeehan (the author here) has some of the best resources on building security programs available on the Internet: https://scrty.io/ https://scrty.io/

encryptluks2 4y ago
I can't fathom why they would then use Medium to publish an article like this instead of their own website.

gsk22 4y ago

> I don’t believe Joe is a criminal, but my personal opinions about his guilt don’t matter. Then why bring it up? > My analysis will treat this as an accident. Isn't the point of an analysis to determine what happened? Not to start with a preconceived idea and make the analysis fit that? It's hard to trust a source repeatedly claiming to be neutral after spending the first few paragraphs espousing their biases.

jasonhansel 4y ago
Usually, in a blameless analysis, you avoid overtly criticizing anyone or assigning blame, but you at least say what different people did. The OP doesn't seem to go into much detail about the exact actions Sullivan took, which seems like a major omission.
- tptacek 4y ago
  It's not about his actions, it's about the processes that enabled them.
  - deleted 4y ago
    [deleted]
deleted 4y ago
[deleted]

your_username 4y ago

jasonhansel 4y ago

My takeaway: whenever private data has inadvertently become available to outside actors, you should treat as a breach, even if you don't have evidence of exfiltration or malicious use, and even if you first found out about the issue from a bug bounty program (legitimate or otherwise).