21 ms·

iPhone Bugs Are Too Valuable to Report to Apple

201 points by cinquemb 9y ago

objclxt 9y ago
But...this is true of every software vendor. Does anyone think Microsoft is paying market value for a remote code execution exploit in Edge? They're not, they'll give you $15k for it[1]. I find this particularly interesting: > [the security researchers] asked Apple's security team for special iPhones that don't have certain restrictions so it's easier to hack them [...] these devices would have some security features, such as sandboxing, disabled in order to allow the researchers to continue doing their work. If I go to Google or Facebook and ask them to, say, turn off some key security features on their site so I can find more bugs they're gong to tell me to go take a hike. It's unclear to me why a security researcher thinks Apple would give them access to a device with the sandbox bypassed. Why would they possibly trust them? [1]: https://technet.microsoft.com/en-us/library/dn425036.aspx https://technet.microsoft.com/en-us/library/dn425036.aspx
- saurik 9y ago
  Your analogy to me falls extremely flat given that in this situation you own the iPhone in question and are just asking for the right to put arbitrary software... on your own phone. (My comment is now at -1. I challenge anyone considering to downvote this response to actually answer how asking for the right to modify the software on the one phone in your possession--the one whose security features, if you really want to insist that this is about a security feature, only affects you--is even remotely comparable in an honest setting to "go[ing] to Google or Facebook and ask[ing] them to, say, turn off some key security features on their site".)
  - andy_ppp 9y ago
    I agree with you here, you can get to other parts of the OS with this ability and will find different flaws. I would say that this comparable to an external penetration test vs a more formal internal practices and code review. Both are acceptable approaches finding different results.
  - ghughes 9y ago
    Many of the phone’s security features don’t only benefit the user - they also make widespread piracy virtually impossible, and they ensure that nobody can set up a rival software ecosystem on Apple’s hardware.
    - saurik 9y ago
      Your comment agrees with my world view (and why I even pointed out that "if you really want to insist..."), but comes across as an attempt to answer my question or refute my point, so I will then respond by saying "and giving a handful of security researchers the ability to run arbitrary software doesn't lead to rampant piracy" (but you know what does? the "free developer" tier added by the Swift team ;P).
  - cageface 9y ago
    This has been my biggest complaint about the iPhone from the very beginning of the app store and remains my primary reservation about committing to Apple devices to this day. Thankfully the UX on Android has caught up to the point where it's as good as or even better, in some respects, than iOS. At least for my purposes, anyway.
  - Someone 9y ago
    I didn't down-vote, and don't know enough of this field to be sure, but I think there's an edge case that can affect other users: opening any door a bit may make it easier to retrieve keys that allows hackers to break into other phones. For example, if they open the secure enclave so that security researchers can see in full detail how it works, that may expose some master key. Countermeasure would be to have separate keys, ideally for each such phone, but having them for only this class of phones might be sufficient, too, if they also restrict access to this class of phones. I think that's technically easy, but may be expensive, process wise.
    - saurik 9y ago
      A system designed like that is flawed and insecure, and someone else with more resources--probably someone much more evil, sadly--is going to have figured out a way to get to that key... we should want to air that to the light of day and get that fixed. I mean, at some level your comment turns into "we should try to hide the security flaws we have as much as possible, even from the people who are actively trying to help the world be more secure"... if you aren't serious about finding bugs, why bother with security?
      - Someone 9y ago
        I don't understand that logic. Apple built a safe. If I want to check that it is secure, it would be helpful if they built one from glass, so that I can see how the mechanism works, and thus verify the design. My argument is that that may expose a secret stored in that safe that then can be used to break into 'real' safes. Your argument seems (to me) to be there should be no secrets on that phone that make attackers any the wiser (might well be possible, given the existence of asymmetric encryption). If so, why did Apple go to the effort to add the secure enclave?
        willstrafach 9y ago
        Saurik is correct about security through obscurity not really being helpful, but for what it is worth, this: > Countermeasure would be to have separate keys, ideally for each such phone, but having them for only this class of phones might be sufficient, too, if they also restrict access to this class of phones. I think that's technically easy, but may be expensive, process wise. ...is already something Apple does with the special development devices. Different keys and certificates used, probably to allow easier access for engineers while tightly monitoring and auditing production infrastructure.
  - jsjohnst 9y ago
    Here's the concern I have, if you're able to "unlock" a single device, how do you do it where it's useful to the researcher, but can't be done to an unsuspecting 3rd party's phone? The only way I can think of that doesn't have an endless array of problematic edge cases is for Apple to have a "researcher edition" of the phone, but even that isn't problem free. Thoughts?
    - saurik 9y ago
      Apple already solves this problem by way of their TSS update server: they can whitelist specific ECIDs to be able to sign and install developer customized firmwares. What the argument was was "if we sign on to your program, can you add us to your whitelist?". The only issue I can come up with would be "what if a researcher who was registered with and known to Apple gave their phone to someone else as a fake gift to spy on them". If you really want to go there we could argue it, but it seems a little far fetched in terms of "amount of damage that can be done via this route that couldn't be done via other ones".
      - jsjohnst 9y ago
        > The only issue I can come up with would be .... If it's locked to specific hardware, then no, I definitely wouldn't argue the edge case scenario you mentioned.
    - willstrafach 9y ago
      Apple has DEV-fused devices which use separate development certificates and keys. The bootloaders and kernels for production devices still retain the code for the "special" functionality, that is how researchers are aware of it, but it simply will not work without an actual dev-fused device.
  - paulcole 9y ago
    You own an iPhone designed to work in a specific way. You don't need Apple's permission to put arbitrary software on your own phone. You can just do it! If you can figure out how and don't mind maybe breaking your iPhone. Apple doesn't sell infinitely customizable devices. Anyone who buys an iPhone knows this going in or is being intentionally obtuse. The idea that it's my phone I should be able to do anything I want with it is completely antithetical to the brand Apple has built over the last 40 years.
    - saurik 9y ago
      I am not certain what conversation you think you are joining, but this is one about "people invited by Apple to take part in a bug bounty program--one with an in-person onboarding process and which involved a meeting with Apple's security team, all expenses paid, in Cupertino--with the stated goal of working together to improve the security of the device have asked to be given the ability to modify the software on a limited number of test devices to aid in their security research". Your response is trying to tackle the general argument for arbitrary customers. I still think you are wrong, but that isn't the argument today.
- michaelbuckbee 9y ago
  You're right, but it's still puzzling to me. That at a certain point, wouldn't it still be cheaper for MS (or whoever) to pay a more serious amount for critical bugs. I don't quite understand the demand side of this, is it just complacency among customers of these companies not demanding more secure systems.
  - TheSpiceIsLife 9y ago
    Wouldn't the black market always be willing to pay more?
    - michaelbuckbee 9y ago
      I guess that's my question. We're talking about companies with hundreds of millions of customers here - which would make me think that any minor security issue's impact is magnified. Which I would think would mean MG/Amz/Goog/Apple would be willing to pay more.
    - qq66 9y ago
      The government markets will always be willing to pay more because money is free to them. In general, the financially motivated criminal black markets would probably be willing to pay less (if hackers use an exploit to steal $250m from Apple users, that probably hurts Apple by more than $250m). However, a $250m bug bounty is hard for Apple to pay, because the harm is hard to quantify, so they generally don't, leaving the criminals to pay $1m for it.
      - PeterisP 9y ago
        Figures like that IMHO indicate that the black market doesn't sell the exploit to teams doing financial fraud (it doesn't pay back that much) but that instead they act as resellers / auction houses for nation states stockpiling cyberweapons.
    - kpcyrd 9y ago
      Wouldn't there be a limit at some point? If apple starts paying 1 million, how many shady companies can afford paying 10 million? Also, if you can make 5 million with a single exploit through bug bounties, would you even bother selling to the blackmarket if you can easily live off of that money for the rest of your life?
- smt88 9y ago
  > Does anyone think Microsoft is paying market value for a remote code execution exploit in Edge? No bug bounty program has to pay market rate. They just have to pay enough so that $bounty > $market_rate - $fear_of_getting_arrested Edit: there are actually other non-monetary benefits to being a white hat. Some of them get high-paying contracts or fantastic jobs. Some just like the prestige of finding flaws. So it's a more complicated equation than I presented above.
  - y_u_no_rust 9y ago
    arrested? there's nothing illegal about selling exploits to non-criminal groups like zerodium or whoever
  - Neliquat 9y ago
    Any evidence of anyone EVER being arrested for selling an exploit or is this pure FUD?
- dsl 9y ago
  > If I go to Google or Facebook and ask them to, say, turn off some key security features on their site so I can find more bugs they're gong to tell me to go take a hike. It isn't that uncommon in bug bounty programs to get specific restrictions (say rate limiting on web apps) turned off. When I do security audits i'll often ask for shortcuts (give me access to X machine as if I phished it, etc). These shortcuts will shave a month of the low level grunt work off so I can spend my limited time working on the core vulnerability.
  - kinkrtyavimoodh 9y ago
    Yes but you are presumably doing these audits on request, not cold-calling them, so to speak.
- masklinn 9y ago
  I did a surface comparison of the Apple, Microsoft and Google programs and while the top lend is the same everywhere (150k~200k) Apple still looks pretty unrewarding when you go below that: Google can pay up to 150k for a critical Android Kernel compromise (not including Trusted Environment compromise) reproducible on a Pixel, Pixel XL or Pixel C. If my reading is correct, at Apple that tops out at 50k, you have to compromise the Secure Enclave to go even reach 100k If I'm not mistaken for MS it's covered under the Mitigation Bypass and Bounty for Defense Program, which is up to 100k + 100k (100k for mitigation bypass, 100k for bounty for defense)
monksy 9y ago
So the short end of the stick, Apple is trying to be stingy on the rewards paid for finding these issues and is complaining that they're not competitive?
- tristanj 9y ago
  It's more like the exploit buyers will always pay more than what Apple would, so if Apple raises its bounties the exploit buyers will raise their amount too. Zerodium could just offer 2x as much as Apples bounty, and Apple can never win on price. It's a losing game for apple to play. What Apple can compete on are the non-monetary incentives, such as prestige, rewards, access, etc.
  - chimeracoder 9y ago
    > It's more like the exploit buyers will always pay more than what Apple would, so if Apple raises its bounties the exploit buyers will raise their amount too. That is not how supply and demand work. (Furthermore, it's also not as simple as exploit buyers paying more, because there are reasons for researchers to sell their exploits to Apple even if other buyers might pay more - the problem described (badly) in the article is that the disparity is currently too high).
    - heartbreak 9y ago
      The demand in this particular marketplace is, at times, extremely high.
  - duncan_bayne 9y ago
    http://www.cnbc.com/2017/05/02/apples-cash-hoard-swells-to-record-256-8-billion.html http://www.cnbc.com/2017/05/02/apples-cash-hoard-swells-to-r... I don't know, but I'm fairly sure, that Apple _could_ outbid the other exploit buyers if they chose.
  - desdiv 9y ago
    >Zerodium could just offer 2x as much as Apples bounty, and Apple can never win on price. I'm a die-hard Android fan, but even I'm forced to admit that iPhone has much better security. And security is a big selling point. Raising the bug bounty to $10 million and it would still be pocket change to Apple, but it would result in: 1. Free advertisement 2. Good PR 3. Outbid the black market for iOS exploits 4. Encourage more white hats to look into iOS security
saurik 9y ago
If you noticed your friend and neighbor left their car unlocked with the keys inside, would you steal their car, or would you tell them? I bet you would tell them. Let's say instead that there were some shady people who liked to hang out outside of the local gas station, and you have heard that they will give you a cut of any heist they pull off based on "tips". Do you call them over to come steal the car? I bet you don't do that either. Let's say that it is some random person's car. I still doubt you steal the car, and I still doubt you tip off someone else to steal the car. However, I also doubt you go far out of your way to find and tell the owner of the car something is wrong. What if, though, you knew that you could get a reward; something sizable enough to be at least worth your time, but nowhere near the value of either the car itself or what a band of thieves would be willing give you if you called them and tipped them off? That's all this bug bounty program is: it is designed to provide a reason for people who come across bugs to even bother coming to Apple at all rather than just putting it in a "pile of fun bugs". Only, instead of the moral issue being "someone's car might get stolen", it is more like "you found a bug in the Tesla's computer locks, which makes it trivial to walk up to any Tesla anywhere and just drive off (or even tell the Tesla to steal itself!)". The companies that offer large sums of cash for key bugs, such as Zerodium, tend to be pretty "black hat"... their clients are doing stuff like corporate and governmental espionage; they might even have mafia-like organizations as clients for all you know. https://www.wired.com/2016/09/top-shelf-iphone-hack-now-goes-1-5-million/ https://www.wired.com/2016/09/top-shelf-iphone-hack-now-goes... So that's the real ethical question involved here: do you go to Apple and get your $50-200,000, knowing that Apple will give you credit for the bug, let you talk about it at the next conference, and seems to care enough to try to fix these things quickly... ...or do you sell your bug to a group that resells it to some government which then uses it to try to spy on people like Ahmed Mansoor, "an internationally recognized human rights defender, based in the United Arab Emirates (UAE), and recipient of the Martin Ennals Award (sometimes referred to as a “Nobel Prize for human rights”)". https://citizenlab.org/2016/08/million-dollar-dissident-iphone-zero-day-nso-group-uae/ https://citizenlab.org/2016/08/million-dollar-dissident-ipho... FWIW, I have severe moral issues with this bug bounty program: I am a strong advocate of simultaneous disclosure, and while Apple does tend to fix bugs quickly, they have made it clear that they are not prepared to commit to timelines even while keeping users in the dark about what they need to do to protect themselves. However, this article makes it sound like the entire concept of the bug bounty program is incompetent or something, as it is failing to pay as much money as the black market... while I have met a few people in the field who are more than happy to sell a bug to literally anyone with cash, the vast majority of people (even the ones whom I have sometimes called "mercenaries" for being willing to "switch sides"), have a pretty serious distaste for the idea of selling a bug to the highest bidder. The real reasons you don't hear much about people selling their bugs to Apple are that they are like Luca (who started doing this at the age of 17--he's now either 19 or 20?--which is context that I think is really important for this evaluation) and are sitting on bugs because they are personally valuable to them (as without at least one bug, you don't even own your own phone enough to look for others; so there's a really big incentive to not disclose your last bug: this is the thing that Apple should really care to fix), that they are intending to release a public weaponized exploit in the form of a jailbreak (which, given the demand from legitimate users due to Apple's insistence on locking down their devices for reasons that are more about business models than security, can be a ticket to world-wide fame that money just can't buy, and which will also net you at least some donations on the side), or simply that they actually have been but they haven't told anyone (a situation that seems so likely that it seems weird that this article discounts it).
PhasmaFelis 9y ago
This article is fucking weird. It's like saying, with a straight face, that the military needs to be competitive with professional-assassin pay scales if they want to hire the best killers.
- qbrass 9y ago
  I'd assume the military would have to pay more than going-rate to hire the professional assassin. It's definitely the wrong time to cheap-out.
- agjacobson 9y ago
  If there's a shortage, or some other performance problem, they do, and they will.
- honestoHeminway 9y ago
  Yes, the sub-system creation of long-term binding contracts (military) is usually not factored into free -market think at all.
microcolonel 9y ago
Well, it's invite only, and some fairly critical exploit categories seem to be way below market value. This is especially silly because these are maximum payouts. Apple is not lacking cash on hand, and while I know that's not a reason to spend it, I figure they could come further toward the going rate. Especially infuriating is the lack of other moral incentives beside "doing the right thing", like when you get a bug bounty for an open source component, and know that the public has free access to it. Even "doing the right thing" when it comes to Apple is morally unrewarding, since they typically treat developers and partners like dirt, and are so isolated from the rest of society. Because of the lack of true community around Apple and their products, owing largely to how proprietary all of their products and programmes are, I don't see why anyone would do white hat security research on their platforms unless they were paid substantially and directly by Apple or an Apple customer. From my perspective, it's enough of a smack in the face to use their products. I doubt many want to be fed what amounts to (relatively speaking) table scraps for elite security research on a platform that you don't own at the end of the day even as a customer. To have to be invited to do this just makes it completely not worth starting if your goal is to participate in the white hat market.
agjacobson 9y ago
1. Sell bug on the open market for whatever price the market will bear. Advantage: paid concomitant to your talents. 2. Wait n days. n could be zero. Don't have a theory for n. 3. Slip the bug to Apple anonymously.
- rubatuga 9y ago
  Well, your payment could be part of a contract where they would only give you the money if Apple doesn't patch the bug in the next n days
  - pluma 9y ago
    Contracts only matter if they are enforceable by law. If a contract involves illegal activity, it is invalid.
    - jsjohnst 9y ago
      There are many kinds of contracts. The contract implied above could be "cross us and find you and your family disappear", not "we sue you in court".
- RoboTeddy 9y ago
  I imagine that whoever paid you a ton of money might not be happy afterwards. I also imagine they're not the type of people you want to make unhappy.
DennisP 9y ago
Apple has $58 billion, and a quarterly gross profit of about $20 billion. Based on the article's numbers, it could outbid the black market by multiplying their payouts by ten. I doubt there are so many bugs that they can't fix this with an insignificant impact to their bottom line, even assuming it doesn't help sales to have a bulletproof reputation for security.
- runeks 9y ago
  I agree. I don't think there's a need to multiply payouts by ten though. When you give regular people -- as opposed to people familiar with selling exploits on the dark web -- the ability to sell exploits legally to Apple, you motivate a much larger group of people. The honest hackers far outnumber exploit vendors if someone is willing to pay them. That someone could be the company that depends on the software for their business (Apple with iOS), or a software insurance company. The very reason companies like e.g. Google, Microsoft, Apple are so profitable is because they have many users, which in turn make exploits against their software very valuable. So, unless a company's profit per user is very small, this should always scale (more users = both higher income from users and higher price for exploits). If they were willing to purchase exploits at market price they would, in effect, be functioning as their own insurer -- in my opinion, this practice is the only sensible thing an insurance company can do to insure software of reasonable complexity. Microsoft paying one million USD per exploit, for 100 high-value exploits per year (0.1bn USD), would decrease their yearly profit (2016: 16.8bn USD) by only 0.6%, but make a huge difference in the security of their software (100 high-value exploits per year is a lot). Stockholders would be foolish to vote no if this were proposed.
  - wepple 9y ago
    > would decrease their yearly profit (2016: 16.8bn USD) by only 0.6% > Stockholders would be foolish to vote no if this were proposed. I doubt that an improvement to security of the type you describe here would have a significant improvement on sales or reputation of MS. So shareholders will see profits go down, who would vote on that?
deleted 9y ago
[deleted]
such_a_casual 9y ago
> It isn't forced on you in any way at all. Uhm, what universe do you live in exactly? Because the universe where I live, I have Apple or Android. Saying that apple's bullshit isn't forced on millions of users is like saying the Republican party's bullshit isn't pushed on millions of citizens. I mean argument aside, let's not do Apple's job for them and paint the world as capitalistic utopia where consumers have all the power and make all the decisions. Consumers do not make 99% of the decisions related to their phone, including the decision to have one.
- princekolt 9y ago
  I don't see how being born into a family of a certain political opinion and being indoctrinated for years can be comparable to choosing a phone brand. I think you are analyzing this question from a very US-centric perspective and failing to realize that in other markets (like Europe and South America, the ones I'm familiar with), Apple has a much smaller footprint, and does not possess the "reach" of a political party at all. Edit: typo
  - saurik 9y ago
    I think the idea is that if you want a high quality phone--and you essentially need a high quality phone to be competitive in the workforce and even to engage in many social activities and functions--you are going to end up buying a device from one of a small handful of companies (Apple, Samsung, Microsoft, Sony, HTC), all of which are closed down and locked experiences. This is a problem that can likely only be solved by legislation (which the EU is thankfully looking into, as the EU actually cares: I <3 the EU).
    - strcat 9y ago
      > all of which are closed down and locked experiences That's not true. Many Android devices have an unlockable bootloader with explicit support for building the Android Open Source Project (AOSP) for the device. Nexus and Pixel devices are directly supported by AOSP without modification. It's the same codebase used to build the stock OS for those device. The stock OS on those devices only adds Google Play apps to the source tree, some of which replace AOSP apps. It doesn't contain any secret sauce changes to AOSP. Android engineers use the same Nexus / Pixel devices that are shipped to consumers as their development devices. You enable OEM unlocking within the OS from the owner account and can then unlock the bootloader via physical access using fastboot over USB, allowing images to be flashed via fastboot. Serial debugging can be toggled on and done via an open source cable design through the headphone port. Other companies like Sony have emulated this by releasing official sources for building AOSP for their unlockable devices rather than only making the bootloader unlockable and leaving it up to the community to hack together support. However, I think it's only Nexus / Pixel devices where you get support for full verified boot with a third party OS (i.e. you can lock the bootloader again, and have it verify the OS using a third party key) along with the ability to toggle on serial debugging. It's why the Android security research community is so active. You get the same sources / build system, development devices (Nexus / Pixel), debugging tools, etc. as an Android engineer working at Google. The only major thing you don't get is access to their internal bug tracker. Hopefully they'll move towards the Chromium model where most of that is public once embargoes are over.
deleted 9y ago
[deleted]
drewmol 9y ago
The article mostly proposes what Apple should do: increase bug bounty payouts. When proposing what the discoverer should do, does it matter WWJD? He's dead so he cannot weigh in, but I think it's fair to suppose early life/career he may sell it to the highest bidder and use the money to invest in a Pixar type, or sell it to both parties if possible. Later life/career possibly use it to negatively impact a competitor? (weighing legal risk to his company of course) How about WW{Apple/Cook}D? I think we know what Uber would do;)

201 points by cinquemb 9y ago

But...this is true of every software vendor. Does anyone think Microsoft is paying market value for a remote code execution exploit in Edge? They're not, they'll give you $15k for it[1]. I find this particularly interesting: > [the security researchers] asked Apple's security team for special iPhones that don't have certain restrictions so it's easier to hack them [...] these devices would have some security features, such as sandboxing, disabled in order to allow the researchers to continue doing their work. If I go to Google or Facebook and ask them to, say, turn off some key security features on their site so I can find more bugs they're gong to tell me to go take a hike. It's unclear to me why a security researcher thinks Apple would give them access to a device with the sandbox bypassed. Why would they possibly trust them? [1]: https://technet.microsoft.com/en-us/library/dn425036.aspx https://technet.microsoft.com/en-us/library/dn425036.aspx

saurik 9y ago
Your analogy to me falls extremely flat given that in this situation you own the iPhone in question and are just asking for the right to put arbitrary software... on your own phone. (My comment is now at -1. I challenge anyone considering to downvote this response to actually answer how asking for the right to modify the software on the one phone in your possession--the one whose security features, if you really want to insist that this is about a security feature, only affects you--is even remotely comparable in an honest setting to "go[ing] to Google or Facebook and ask[ing] them to, say, turn off some key security features on their site".)
- andy_ppp 9y ago
  I agree with you here, you can get to other parts of the OS with this ability and will find different flaws. I would say that this comparable to an external penetration test vs a more formal internal practices and code review. Both are acceptable approaches finding different results.
- ghughes 9y ago
  Many of the phone’s security features don’t only benefit the user - they also make widespread piracy virtually impossible, and they ensure that nobody can set up a rival software ecosystem on Apple’s hardware.
  - saurik 9y ago
    Your comment agrees with my world view (and why I even pointed out that "if you really want to insist..."), but comes across as an attempt to answer my question or refute my point, so I will then respond by saying "and giving a handful of security researchers the ability to run arbitrary software doesn't lead to rampant piracy" (but you know what does? the "free developer" tier added by the Swift team ;P).
- cageface 9y ago
  This has been my biggest complaint about the iPhone from the very beginning of the app store and remains my primary reservation about committing to Apple devices to this day. Thankfully the UX on Android has caught up to the point where it's as good as or even better, in some respects, than iOS. At least for my purposes, anyway.
- Someone 9y ago
  I didn't down-vote, and don't know enough of this field to be sure, but I think there's an edge case that can affect other users: opening any door a bit may make it easier to retrieve keys that allows hackers to break into other phones. For example, if they open the secure enclave so that security researchers can see in full detail how it works, that may expose some master key. Countermeasure would be to have separate keys, ideally for each such phone, but having them for only this class of phones might be sufficient, too, if they also restrict access to this class of phones. I think that's technically easy, but may be expensive, process wise.
  - saurik 9y ago
    A system designed like that is flawed and insecure, and someone else with more resources--probably someone much more evil, sadly--is going to have figured out a way to get to that key... we should want to air that to the light of day and get that fixed. I mean, at some level your comment turns into "we should try to hide the security flaws we have as much as possible, even from the people who are actively trying to help the world be more secure"... if you aren't serious about finding bugs, why bother with security?
    - Someone 9y ago
      I don't understand that logic. Apple built a safe. If I want to check that it is secure, it would be helpful if they built one from glass, so that I can see how the mechanism works, and thus verify the design. My argument is that that may expose a secret stored in that safe that then can be used to break into 'real' safes. Your argument seems (to me) to be there should be no secrets on that phone that make attackers any the wiser (might well be possible, given the existence of asymmetric encryption). If so, why did Apple go to the effort to add the secure enclave?
      - willstrafach 9y ago
        Saurik is correct about security through obscurity not really being helpful, but for what it is worth, this: > Countermeasure would be to have separate keys, ideally for each such phone, but having them for only this class of phones might be sufficient, too, if they also restrict access to this class of phones. I think that's technically easy, but may be expensive, process wise. ...is already something Apple does with the special development devices. Different keys and certificates used, probably to allow easier access for engineers while tightly monitoring and auditing production infrastructure.
- jsjohnst 9y ago
  Here's the concern I have, if you're able to "unlock" a single device, how do you do it where it's useful to the researcher, but can't be done to an unsuspecting 3rd party's phone? The only way I can think of that doesn't have an endless array of problematic edge cases is for Apple to have a "researcher edition" of the phone, but even that isn't problem free. Thoughts?
  - saurik 9y ago
    Apple already solves this problem by way of their TSS update server: they can whitelist specific ECIDs to be able to sign and install developer customized firmwares. What the argument was was "if we sign on to your program, can you add us to your whitelist?". The only issue I can come up with would be "what if a researcher who was registered with and known to Apple gave their phone to someone else as a fake gift to spy on them". If you really want to go there we could argue it, but it seems a little far fetched in terms of "amount of damage that can be done via this route that couldn't be done via other ones".
    - jsjohnst 9y ago
      > The only issue I can come up with would be .... If it's locked to specific hardware, then no, I definitely wouldn't argue the edge case scenario you mentioned.
  - willstrafach 9y ago
    Apple has DEV-fused devices which use separate development certificates and keys. The bootloaders and kernels for production devices still retain the code for the "special" functionality, that is how researchers are aware of it, but it simply will not work without an actual dev-fused device.
- paulcole 9y ago
  You own an iPhone designed to work in a specific way. You don't need Apple's permission to put arbitrary software on your own phone. You can just do it! If you can figure out how and don't mind maybe breaking your iPhone. Apple doesn't sell infinitely customizable devices. Anyone who buys an iPhone knows this going in or is being intentionally obtuse. The idea that it's my phone I should be able to do anything I want with it is completely antithetical to the brand Apple has built over the last 40 years.
  - saurik 9y ago
    I am not certain what conversation you think you are joining, but this is one about "people invited by Apple to take part in a bug bounty program--one with an in-person onboarding process and which involved a meeting with Apple's security team, all expenses paid, in Cupertino--with the stated goal of working together to improve the security of the device have asked to be given the ability to modify the software on a limited number of test devices to aid in their security research". Your response is trying to tackle the general argument for arbitrary customers. I still think you are wrong, but that isn't the argument today.
michaelbuckbee 9y ago
You're right, but it's still puzzling to me. That at a certain point, wouldn't it still be cheaper for MS (or whoever) to pay a more serious amount for critical bugs. I don't quite understand the demand side of this, is it just complacency among customers of these companies not demanding more secure systems.
- TheSpiceIsLife 9y ago
  Wouldn't the black market always be willing to pay more?
  - michaelbuckbee 9y ago
    I guess that's my question. We're talking about companies with hundreds of millions of customers here - which would make me think that any minor security issue's impact is magnified. Which I would think would mean MG/Amz/Goog/Apple would be willing to pay more.
  - qq66 9y ago
    The government markets will always be willing to pay more because money is free to them. In general, the financially motivated criminal black markets would probably be willing to pay less (if hackers use an exploit to steal $250m from Apple users, that probably hurts Apple by more than $250m). However, a $250m bug bounty is hard for Apple to pay, because the harm is hard to quantify, so they generally don't, leaving the criminals to pay $1m for it.
    - PeterisP 9y ago
      Figures like that IMHO indicate that the black market doesn't sell the exploit to teams doing financial fraud (it doesn't pay back that much) but that instead they act as resellers / auction houses for nation states stockpiling cyberweapons.
  - kpcyrd 9y ago
    Wouldn't there be a limit at some point? If apple starts paying 1 million, how many shady companies can afford paying 10 million? Also, if you can make 5 million with a single exploit through bug bounties, would you even bother selling to the blackmarket if you can easily live off of that money for the rest of your life?
smt88 9y ago
> Does anyone think Microsoft is paying market value for a remote code execution exploit in Edge? No bug bounty program has to pay market rate. They just have to pay enough so that $bounty > $market_rate - $fear_of_getting_arrested Edit: there are actually other non-monetary benefits to being a white hat. Some of them get high-paying contracts or fantastic jobs. Some just like the prestige of finding flaws. So it's a more complicated equation than I presented above.
- y_u_no_rust 9y ago
  arrested? there's nothing illegal about selling exploits to non-criminal groups like zerodium or whoever
- Neliquat 9y ago
  Any evidence of anyone EVER being arrested for selling an exploit or is this pure FUD?
dsl 9y ago
> If I go to Google or Facebook and ask them to, say, turn off some key security features on their site so I can find more bugs they're gong to tell me to go take a hike. It isn't that uncommon in bug bounty programs to get specific restrictions (say rate limiting on web apps) turned off. When I do security audits i'll often ask for shortcuts (give me access to X machine as if I phished it, etc). These shortcuts will shave a month of the low level grunt work off so I can spend my limited time working on the core vulnerability.
- kinkrtyavimoodh 9y ago
  Yes but you are presumably doing these audits on request, not cold-calling them, so to speak.
masklinn 9y ago
I did a surface comparison of the Apple, Microsoft and Google programs and while the top lend is the same everywhere (150k~200k) Apple still looks pretty unrewarding when you go below that: Google can pay up to 150k for a critical Android Kernel compromise (not including Trusted Environment compromise) reproducible on a Pixel, Pixel XL or Pixel C. If my reading is correct, at Apple that tops out at 50k, you have to compromise the Secure Enclave to go even reach 100k If I'm not mistaken for MS it's covered under the Mitigation Bypass and Bounty for Defense Program, which is up to 100k + 100k (100k for mitigation bypass, 100k for bounty for defense)

monksy 9y ago

So the short end of the stick, Apple is trying to be stingy on the rewards paid for finding these issues and is complaining that they're not competitive?

tristanj 9y ago
It's more like the exploit buyers will always pay more than what Apple would, so if Apple raises its bounties the exploit buyers will raise their amount too. Zerodium could just offer 2x as much as Apples bounty, and Apple can never win on price. It's a losing game for apple to play. What Apple can compete on are the non-monetary incentives, such as prestige, rewards, access, etc.
- chimeracoder 9y ago
  > It's more like the exploit buyers will always pay more than what Apple would, so if Apple raises its bounties the exploit buyers will raise their amount too. That is not how supply and demand work. (Furthermore, it's also not as simple as exploit buyers paying more, because there are reasons for researchers to sell their exploits to Apple even if other buyers might pay more - the problem described (badly) in the article is that the disparity is currently too high).
  - heartbreak 9y ago
    The demand in this particular marketplace is, at times, extremely high.
- duncan_bayne 9y ago
  http://www.cnbc.com/2017/05/02/apples-cash-hoard-swells-to-record-256-8-billion.html http://www.cnbc.com/2017/05/02/apples-cash-hoard-swells-to-r... I don't know, but I'm fairly sure, that Apple _could_ outbid the other exploit buyers if they chose.
- desdiv 9y ago
  >Zerodium could just offer 2x as much as Apples bounty, and Apple can never win on price. I'm a die-hard Android fan, but even I'm forced to admit that iPhone has much better security. And security is a big selling point. Raising the bug bounty to $10 million and it would still be pocket change to Apple, but it would result in: 1. Free advertisement 2. Good PR 3. Outbid the black market for iOS exploits 4. Encourage more white hats to look into iOS security

saurik 9y ago

If you noticed your friend and neighbor left their car unlocked with the keys inside, would you steal their car, or would you tell them? I bet you would tell them. Let's say instead that there were some shady people who liked to hang out outside of the local gas station, and you have heard that they will give you a cut of any heist they pull off based on "tips". Do you call them over to come steal the car? I bet you don't do that either. Let's say that it is some random person's car. I still doubt you steal the car, and I still doubt you tip off someone else to steal the car. However, I also doubt you go far out of your way to find and tell the owner of the car something is wrong. What if, though, you knew that you could get a reward; something sizable enough to be at least worth your time, but nowhere near the value of either the car itself or what a band of thieves would be willing give you if you called them and tipped them off? That's all this bug bounty program is: it is designed to provide a reason for people who come across bugs to even bother coming to Apple at all rather than just putting it in a "pile of fun bugs". Only, instead of the moral issue being "someone's car might get stolen", it is more like "you found a bug in the Tesla's computer locks, which makes it trivial to walk up to any Tesla anywhere and just drive off (or even tell the Tesla to steal itself!)". The companies that offer large sums of cash for key bugs, such as Zerodium, tend to be pretty "black hat"... their clients are doing stuff like corporate and governmental espionage; they might even have mafia-like organizations as clients for all you know. https://www.wired.com/2016/09/top-shelf-iphone-hack-now-goes-1-5-million/ https://www.wired.com/2016/09/top-shelf-iphone-hack-now-goes... So that's the real ethical question involved here: do you go to Apple and get your $50-200,000, knowing that Apple will give you credit for the bug, let you talk about it at the next conference, and seems to care enough to try to fix these things quickly... ...or do you sell your bug to a group that resells it to some government which then uses it to try to spy on people like Ahmed Mansoor, "an internationally recognized human rights defender, based in the United Arab Emirates (UAE), and recipient of the Martin Ennals Award (sometimes referred to as a “Nobel Prize for human rights”)". https://citizenlab.org/2016/08/million-dollar-dissident-iphone-zero-day-nso-group-uae/ https://citizenlab.org/2016/08/million-dollar-dissident-ipho... FWIW, I have severe moral issues with this bug bounty program: I am a strong advocate of simultaneous disclosure, and while Apple does tend to fix bugs quickly, they have made it clear that they are not prepared to commit to timelines even while keeping users in the dark about what they need to do to protect themselves. However, this article makes it sound like the entire concept of the bug bounty program is incompetent or something, as it is failing to pay as much money as the black market... while I have met a few people in the field who are more than happy to sell a bug to literally anyone with cash, the vast majority of people (even the ones whom I have sometimes called "mercenaries" for being willing to "switch sides"), have a pretty serious distaste for the idea of selling a bug to the highest bidder. The real reasons you don't hear much about people selling their bugs to Apple are that they are like Luca (who started doing this at the age of 17--he's now either 19 or 20?--which is context that I think is really important for this evaluation) and are sitting on bugs because they are personally valuable to them (as without at least one bug, you don't even own your own phone enough to look for others; so there's a really big incentive to not disclose your last bug: this is the thing that Apple should really care to fix), that they are intending to release a public weaponized exploit in the form of a jailbreak (which, given the demand from legitimate users due to Apple's insistence on locking down their devices for reasons that are more about business models than security, can be a ticket to world-wide fame that money just can't buy, and which will also net you at least some donations on the side), or simply that they actually have been but they haven't told anyone (a situation that seems so likely that it seems weird that this article discounts it).

PhasmaFelis 9y ago

This article is fucking weird. It's like saying, with a straight face, that the military needs to be competitive with professional-assassin pay scales if they want to hire the best killers.

qbrass 9y ago
I'd assume the military would have to pay more than going-rate to hire the professional assassin. It's definitely the wrong time to cheap-out.
agjacobson 9y ago
If there's a shortage, or some other performance problem, they do, and they will.
honestoHeminway 9y ago
Yes, the sub-system creation of long-term binding contracts (military) is usually not factored into free -market think at all.

microcolonel 9y ago

Well, it's invite only, and some fairly critical exploit categories seem to be way below market value. This is especially silly because these are maximum payouts. Apple is not lacking cash on hand, and while I know that's not a reason to spend it, I figure they could come further toward the going rate. Especially infuriating is the lack of other moral incentives beside "doing the right thing", like when you get a bug bounty for an open source component, and know that the public has free access to it. Even "doing the right thing" when it comes to Apple is morally unrewarding, since they typically treat developers and partners like dirt, and are so isolated from the rest of society. Because of the lack of true community around Apple and their products, owing largely to how proprietary all of their products and programmes are, I don't see why anyone would do white hat security research on their platforms unless they were paid substantially and directly by Apple or an Apple customer. From my perspective, it's enough of a smack in the face to use their products. I doubt many want to be fed what amounts to (relatively speaking) table scraps for elite security research on a platform that you don't own at the end of the day even as a customer. To have to be invited to do this just makes it completely not worth starting if your goal is to participate in the white hat market.

agjacobson 9y ago

1. Sell bug on the open market for whatever price the market will bear. Advantage: paid concomitant to your talents. 2. Wait n days. n could be zero. Don't have a theory for n. 3. Slip the bug to Apple anonymously.

rubatuga 9y ago
Well, your payment could be part of a contract where they would only give you the money if Apple doesn't patch the bug in the next n days
- pluma 9y ago
  Contracts only matter if they are enforceable by law. If a contract involves illegal activity, it is invalid.
  - jsjohnst 9y ago
    There are many kinds of contracts. The contract implied above could be "cross us and find you and your family disappear", not "we sue you in court".
RoboTeddy 9y ago
I imagine that whoever paid you a ton of money might not be happy afterwards. I also imagine they're not the type of people you want to make unhappy.

DennisP 9y ago

Apple has $58 billion, and a quarterly gross profit of about $20 billion. Based on the article's numbers, it could outbid the black market by multiplying their payouts by ten. I doubt there are so many bugs that they can't fix this with an insignificant impact to their bottom line, even assuming it doesn't help sales to have a bulletproof reputation for security.

runeks 9y ago
I agree. I don't think there's a need to multiply payouts by ten though. When you give regular people -- as opposed to people familiar with selling exploits on the dark web -- the ability to sell exploits legally to Apple, you motivate a much larger group of people. The honest hackers far outnumber exploit vendors if someone is willing to pay them. That someone could be the company that depends on the software for their business (Apple with iOS), or a software insurance company. The very reason companies like e.g. Google, Microsoft, Apple are so profitable is because they have many users, which in turn make exploits against their software very valuable. So, unless a company's profit per user is very small, this should always scale (more users = both higher income from users and higher price for exploits). If they were willing to purchase exploits at market price they would, in effect, be functioning as their own insurer -- in my opinion, this practice is the only sensible thing an insurance company can do to insure software of reasonable complexity. Microsoft paying one million USD per exploit, for 100 high-value exploits per year (0.1bn USD), would decrease their yearly profit (2016: 16.8bn USD) by only 0.6%, but make a huge difference in the security of their software (100 high-value exploits per year is a lot). Stockholders would be foolish to vote no if this were proposed.
- wepple 9y ago
  > would decrease their yearly profit (2016: 16.8bn USD) by only 0.6% > Stockholders would be foolish to vote no if this were proposed. I doubt that an improvement to security of the type you describe here would have a significant improvement on sales or reputation of MS. So shareholders will see profits go down, who would vote on that?

deleted 9y ago

[deleted]

such_a_casual 9y ago

> It isn't forced on you in any way at all. Uhm, what universe do you live in exactly? Because the universe where I live, I have Apple or Android. Saying that apple's bullshit isn't forced on millions of users is like saying the Republican party's bullshit isn't pushed on millions of citizens. I mean argument aside, let's not do Apple's job for them and paint the world as capitalistic utopia where consumers have all the power and make all the decisions. Consumers do not make 99% of the decisions related to their phone, including the decision to have one.

princekolt 9y ago
I don't see how being born into a family of a certain political opinion and being indoctrinated for years can be comparable to choosing a phone brand. I think you are analyzing this question from a very US-centric perspective and failing to realize that in other markets (like Europe and South America, the ones I'm familiar with), Apple has a much smaller footprint, and does not possess the "reach" of a political party at all. Edit: typo
- saurik 9y ago
  I think the idea is that if you want a high quality phone--and you essentially need a high quality phone to be competitive in the workforce and even to engage in many social activities and functions--you are going to end up buying a device from one of a small handful of companies (Apple, Samsung, Microsoft, Sony, HTC), all of which are closed down and locked experiences. This is a problem that can likely only be solved by legislation (which the EU is thankfully looking into, as the EU actually cares: I <3 the EU).
  - strcat 9y ago
    > all of which are closed down and locked experiences That's not true. Many Android devices have an unlockable bootloader with explicit support for building the Android Open Source Project (AOSP) for the device. Nexus and Pixel devices are directly supported by AOSP without modification. It's the same codebase used to build the stock OS for those device. The stock OS on those devices only adds Google Play apps to the source tree, some of which replace AOSP apps. It doesn't contain any secret sauce changes to AOSP. Android engineers use the same Nexus / Pixel devices that are shipped to consumers as their development devices. You enable OEM unlocking within the OS from the owner account and can then unlock the bootloader via physical access using fastboot over USB, allowing images to be flashed via fastboot. Serial debugging can be toggled on and done via an open source cable design through the headphone port. Other companies like Sony have emulated this by releasing official sources for building AOSP for their unlockable devices rather than only making the bootloader unlockable and leaving it up to the community to hack together support. However, I think it's only Nexus / Pixel devices where you get support for full verified boot with a third party OS (i.e. you can lock the bootloader again, and have it verify the OS using a third party key) along with the ability to toggle on serial debugging. It's why the Android security research community is so active. You get the same sources / build system, development devices (Nexus / Pixel), debugging tools, etc. as an Android engineer working at Google. The only major thing you don't get is access to their internal bug tracker. Hopefully they'll move towards the Chromium model where most of that is public once embargoes are over.

deleted 9y ago

[deleted]

drewmol 9y ago

The article mostly proposes what Apple should do: increase bug bounty payouts. When proposing what the discoverer should do, does it matter WWJD? He's dead so he cannot weigh in, but I think it's fair to suppose early life/career he may sell it to the highest bidder and use the money to invest in a Pixar type, or sell it to both parties if possible. Later life/career possibly use it to negatively impact a competitor? (weighing legal risk to his company of course) How about WW{Apple/Cook}D? I think we know what Uber would do;)