4 ms·

could you try litle bit harder to provide any example why it is "harder than it looks". you repeated multiple times that its hard, but what exactly(aproximately

could you try litle bit harder to provide any example why it is "harder than it looks". you repeated multiple times that its hard, but what exactly(aproximately) makes it hard?

OJFord 5y ago
I think it's the phrases 'some coordination' and 'company the size of Apple'. It's rarely the case (well, hopefully?!) that a fix is as trivial as 'oh yeah, oops, let's delete that `leak_data()` line' - it's going to involve multiple teams and they're all going to think anything from 'nothing to do with us' to 'hm yes I can see how that happened, but what we're doing in our piece of the pie is correct/needs to be so, this will need to be handled by the [other] team'. Not to say that people 'pass the buck' or are not taking responsibility exactly, just that teams can be insular, and they're all going to view it from the perspective of their side of the 'API', and not find a problem. (Of course with a strict actual API that couldn't be the case, but I use it here only loosely or analogously.) Someone has to coordinate them all, and ultimately probably work out (or decide somewhat arbitrarily - at least in technical terms, but perhaps on the basis of cost or complexity or politics or cetera) whose problem to make it.
- sam0x17 5y ago
  What's worse, typically an exploit doesn't involve knowledge of the actual line of code responsible -- it's just a vague description of behavior or actions that leads to an exploit, making it much easier to pass the buck in terms of who is actually responsible for fixing it. The kicker is if your department/project/whatever fixes it, you're also taking responsibility for causing this error / causing this huge affront to the Apple way...
  - saagarjha 5y ago
    Most good exploits have a pretty solid root cause attached to them.
jcims 5y ago
It's mostly just 'human factors'. What I'm describing below applies across the spectrum of bug reports from fake to huge to everything in between. Nothing of what I'm listing below is an attempt to directly explain or rationalize events in the article, it's just some context from my (anecdotal) experience. - The security researcher community is composed of a broad spectrum of people. Most of them are amazing. However, there is a portion of complete assholes. They send in lazy work, claim bugs like scalps and get super aggressive privately and publicly if things don't go their way or bugs don't get fixed fast enough or someone questions their claims (particularly about severity). This grates on everybody in the chain from the triagers to the product engineers. - Some bounty programs are horribly run. They drop the ball constantly, ignore reports, drag fixes out for months and months, undercut severity...all of which impact payout to the researcher. These stories get a lot of traction in the community, diminishing trust in the model and exacerbating the previous point because nobody wants to be had. - Bug bounties create financial incentives to report bugs, which means that you get a lot of bullshit reports to wade through and catastrophization of even the smallest issues. (Check out @CluelessSec aka BugBountyKing on twitter for parodies but kind of not) This reduces SnR and allows actual major issues to sit around because at first glance they aren't always distinguishable from garbage. - In large orgs, bug bounties are typically run through the infosec and/or risk part of the organization. Their interface to the affected product teams is going to generally be through product owners and/or existing vulnerability reporting mechanisms. Sometimes this is complicated by subsidiary relationships and/or outsourced development. In any case, these bugs will enter the pool of broader security bugs that have been identified through internal scanning tools, security assessments, pen tests and other reports. Just because someone reported them from the outside doesn't mean they get moved to the top of the priority heap. - Again in most cases, product owns the bug. Which means that even though it has been triaged, the product team generally still has a lot of discretion about what to do with it. If its a major issue and the product team stalls then you end up with major escalations through EVP/legal channels. These conversations can get heated. - The bugs themselves often lack context and are randomly distributed through the codebase. Most of the time the development teams are busy cranking out new features, burning down tech debt or otherwise have their focus directed to specific parts of the product. They are used to getting things like static analysis reports saying 'hey commit you just sent through could have a sql injection' and fixing it without skipping a beat (or more likely showing its a false positive). When bug reports come in from the outside, the code underlying the issue may have not been touched for literally years, the teams that built it could be gone, and it could be slated for replacement in the next quarter. - Some of the bugs people find are actually hard to solve and/or the people in the product teams don't fully understand the mechanism of action and put in place basic countermeasures that are easily defeated. This exacerbates the problem, especially if there's an asshole researcher on the other end of the line that just goes out and immediately starts dunking on them on social media. - Most bugs are just simple human error and the teams surrounding the person that did the commit are going to typically want to come to their defense just out of camaraderie and empathy. This is going to have a net chilling effect on barn burners that come through because people don't want to burn their buddies at the stake. All of this to say it takes a lot of culture tending and diplomacy on the part of the bounty runners to manage these factors while trying to make sure each side lives up to their end of the bargain. Most of running a bounty is administrative and applied technical security skills, this part is not...which is why I said it can be the hardest.

could you try litle bit harder to provide any example why it is "harder than it looks". you repeated multiple times that its hard, but what exactly(aproximately) makes it hard?

OJFord 5y ago

I think it's the phrases 'some coordination' and 'company the size of Apple'. It's rarely the case (well, hopefully?!) that a fix is as trivial as 'oh yeah, oops, let's delete that `leak_data()` line' - it's going to involve multiple teams and they're all going to think anything from 'nothing to do with us' to 'hm yes I can see how that happened, but what we're doing in our piece of the pie is correct/needs to be so, this will need to be handled by the [other] team'. Not to say that people 'pass the buck' or are not taking responsibility exactly, just that teams can be insular, and they're all going to view it from the perspective of their side of the 'API', and not find a problem. (Of course with a strict actual API that couldn't be the case, but I use it here only loosely or analogously.) Someone has to coordinate them all, and ultimately probably work out (or decide somewhat arbitrarily - at least in technical terms, but perhaps on the basis of cost or complexity or politics or cetera) whose problem to make it.

sam0x17 5y ago
What's worse, typically an exploit doesn't involve knowledge of the actual line of code responsible -- it's just a vague description of behavior or actions that leads to an exploit, making it much easier to pass the buck in terms of who is actually responsible for fixing it. The kicker is if your department/project/whatever fixes it, you're also taking responsibility for causing this error / causing this huge affront to the Apple way...
- saagarjha 5y ago
  Most good exploits have a pretty solid root cause attached to them.

jcims 5y ago

It's mostly just 'human factors'. What I'm describing below applies across the spectrum of bug reports from fake to huge to everything in between. Nothing of what I'm listing below is an attempt to directly explain or rationalize events in the article, it's just some context from my (anecdotal) experience. - The security researcher community is composed of a broad spectrum of people. Most of them are amazing. However, there is a portion of complete assholes. They send in lazy work, claim bugs like scalps and get super aggressive privately and publicly if things don't go their way or bugs don't get fixed fast enough or someone questions their claims (particularly about severity). This grates on everybody in the chain from the triagers to the product engineers. - Some bounty programs are horribly run. They drop the ball constantly, ignore reports, drag fixes out for months and months, undercut severity...all of which impact payout to the researcher. These stories get a lot of traction in the community, diminishing trust in the model and exacerbating the previous point because nobody wants to be had. - Bug bounties create financial incentives to report bugs, which means that you get a lot of bullshit reports to wade through and catastrophization of even the smallest issues. (Check out @CluelessSec aka BugBountyKing on twitter for parodies but kind of not) This reduces SnR and allows actual major issues to sit around because at first glance they aren't always distinguishable from garbage. - In large orgs, bug bounties are typically run through the infosec and/or risk part of the organization. Their interface to the affected product teams is going to generally be through product owners and/or existing vulnerability reporting mechanisms. Sometimes this is complicated by subsidiary relationships and/or outsourced development. In any case, these bugs will enter the pool of broader security bugs that have been identified through internal scanning tools, security assessments, pen tests and other reports. Just because someone reported them from the outside doesn't mean they get moved to the top of the priority heap. - Again in most cases, product owns the bug. Which means that even though it has been triaged, the product team generally still has a lot of discretion about what to do with it. If its a major issue and the product team stalls then you end up with major escalations through EVP/legal channels. These conversations can get heated. - The bugs themselves often lack context and are randomly distributed through the codebase. Most of the time the development teams are busy cranking out new features, burning down tech debt or otherwise have their focus directed to specific parts of the product. They are used to getting things like static analysis reports saying 'hey commit you just sent through could have a sql injection' and fixing it without skipping a beat (or more likely showing its a false positive). When bug reports come in from the outside, the code underlying the issue may have not been touched for literally years, the teams that built it could be gone, and it could be slated for replacement in the next quarter. - Some of the bugs people find are actually hard to solve and/or the people in the product teams don't fully understand the mechanism of action and put in place basic countermeasures that are easily defeated. This exacerbates the problem, especially if there's an asshole researcher on the other end of the line that just goes out and immediately starts dunking on them on social media. - Most bugs are just simple human error and the teams surrounding the person that did the commit are going to typically want to come to their defense just out of camaraderie and empathy. This is going to have a net chilling effect on barn burners that come through because people don't want to burn their buddies at the stake. All of this to say it takes a lot of culture tending and diplomacy on the part of the bounty runners to manage these factors while trying to make sure each side lives up to their end of the bargain. Most of running a bounty is administrative and applied technical security skills, this part is not...which is why I said it can be the hardest.