11 ms·

Apple’s device surveillance plan is a threat to user privacy – and press freedom

310 points by bangonkeyboard 5y ago

encryptluks2 5y ago
There business model has always been a threat to user privacy and press freedom, many people were just okay with it cause Apple told them otherwise. Open source is the only model where you can verify though, and so Apple was about blind trust and never about privacy.
- 99mans 5y ago
  Why is this being downvoted? Depressing to see that open source software is not respected here.
  - sandworm101 5y ago
    People downvote things that make them angry. The things that make people most angry are those things that strike closest to home. An incorrect or unacceptable comment will be downvoted and commented on, even reported. A perfectly valid and true statement, one that is so true that it scares people, will be silently downvoted. It is a form of denial, an attempt to remove a painful truth.
  - simondotau 5y ago
    I didn't downvote it, but I can understand why some people might have. Because while it's superficially true, it's disingenuous in practice. All security is, at its core, a network of trust between you and other entities. Open source isn't secure because it's open, it's secure because you trust Canonical, you trust Linus Torvalds, you trust GNU, etc. And those people trust other people—hence the "network" of trust. Saying that you can inspect the source code is true in theory. But unless you've done the full audit yourself, you're personally as blind as you are with closed source code. You're choosing to trust whichever security researchers deeply understand the security implications of all the gobbledegook in all the USB drivers, and you're trusting that Canonical is shipping you the same version that security researchers have validated. For 99.9% of users, it all comes down to blind trust. As Linus himself once said: "If you have ever done any security work and it didn't involve the concept of a network of trust, it was not a security work. It was masturbation."
- npteljes 5y ago
  I agree, this is what's to be expected when people rely on proprietary and other people's computers. Stallman's essays, among others, depicted scenarios like this for ages. But people don't like to miss the latest shiny, the convenience of the smart phone tech, so, here we are. They also don't like a told-you-so.
deleted 5y ago
[deleted]
fmajid 5y ago
Of course. Politicians and officials hate whistleblowers far more than pedophiles, and it's obvious this mechanism will be used to find future Chelsea Mannings and Reality Winners with better opsec to make examples out of them.
- HNPoliticalBias 5y ago
  Any population that allows themselves to be ruled by tyrants, will be, as we're seeing with the West slowly morphing into the USSR.
- sparker72678 5y ago
  How? I'd love to be convinced that this mechanism could be easily extended to any content at any time without it being totally obvious to the rest of us that it's being used that way.
  - jjcon 5y ago
    Not OP and not speaking with any authority on this more just curious: If alongside this Apple rolled out the ability for them to initiate a scan on any phone for any thing (not a tool for mass surveillance but targeted surveillance) how would we know given Apples closed ecosystem? This doesn't necessarily speak to the issue here but more to the Apple ethos in general, but I am curious.
    - simondotau 5y ago
      This twitter thread is a useful read: https://twitter.com/pwnallthethings/status/1424873629003702273 https://twitter.com/pwnallthethings/status/14248736290037022...
      - techdragon 5y ago
        Very good breakdown of the situation. But cold comfort for anyone not in the USA. Because we know once Apple ships this it will be expanded to other countries with similar CSAM laws … many of which do not have robust Fourth Amendment type rights enshrined in their legal systems.
threeseed 5y ago
This article is full of inaccuracies. a) People have extracted a pre-release version from an older iOS phone. It is not the one that will be released and not the same one Apple uses server-side to verify the process. b) Adversaries can not reasonably break this process by flooding it with bad data across a range of compromised phones. The client side version is there to prevent this as is the fact it would require jailbroken devices with iCloud logins. c) If you can't trust Apple not to include non-CSAM hashes in the database then how can you trust them not to compromise the operating system itself ? It's illogical. Apple verifies hashes against two sources and two manual checks are required thus needing multiple failure points in the process.
- mattnewton 5y ago
  For b), I can't see how creating spurious icloud accounts and spoofing it will be hard at it's face. I have made dozens of icloud accounts personally for testing in the past. They may take steps to address this attack vector, it's not an unsolvable problem but it isn't solved by requiring a device or icloud account as far as I can tell. For c), the problem isn't just trusting Apple, for whom the hashes are somewhat opaque, my understanding is that they are provided by a governmentally regulated third party.
  - kemayo 5y ago
    For c) you still need to imagine a scenario where Apple is deliberately complicit, because of the human review step. If non-CSAM hashes are inserted into the database, the human reviewers need to know what non-CSAM they're looking for -- otherwise they'll see the e.g. picture of a leaked document, observe that it's not child porn, and flag it as a false positive.
    - intricatedetail 5y ago
      How would they know it is a false positive and not a CSAM image concealed in a document image that NeuralHash was able to "see"? It will be impossible to tell from a lowres picture with certainty.
mrharrison 5y ago
There is an easy way to cast your vote for saying yes to Privacy. Turn off auto-updates and don't update to iOS 15. Spread the word.
- hypothesis 5y ago
  Presumably, there will be a parallel track to stay on iOS 14 with security updates, at least for some time.
  - SSLy 5y ago
    This tech is in iOS since 14.3
    - hypothesis 5y ago
      The neuralHash part, not the whole CSAM scanning daemon/pipeline, right? Security updates to iOS 14 will let people (who might not be ready/able to switch from iOS) to actually do what GP said, without unnecessarily exposing themselves to security bugs in outdated OS.
  - OneLeggedCat 5y ago
    If enough people do that, then no reason that they won't just enable it in 14 too.
    - hypothesis 5y ago
      Assuming that I was not gaslit by other people, I was under impression that would at least require some change to EULA. Otherwise it sounds like a nuclear option by Apple, with dire effects.
      - greyface- 5y ago
        iOS 14 EULA: > By using the Apple Software, you agree that Apple may download and install automatic Apple Software Updates onto your Device and your peripheral devices. > Apple and its licensors reserve the right to change, suspend, remove, or disable access to any Services at any time without notice.
jeremygaither 5y ago
I agree with the article that not only are implementation problems a potential threat, but it also sets a dangerous precedent. I also understand that Apple is attempting to compromise with governments that are still pressuring it to create an encryption back door or hand over keys (which it did in China by having another company operate iCloud there). Governments have often used the pursuit and prosecution of child abusers and terrorists as a scapegoat for eroding privacy for all its citizens. Privacy and liberty versus the lawful pursuit of criminals are challenging problems to balance, especially when governments and law enforcement want a free pass searching for whatever content they deem illegal. Governments decide what photos, communications, and even thoughts (if they could) are legal and which one are illegal. Who knows what some (new) governments may decree new obscenity laws? They could legislate that some pictures once considered lawful and innocent are now unlawful and obscene, retroactively, and ask or attempt to force Apple to search for those newly criminal images. Apple’s new image scanning technology is almost as much of a backdoor to individuals' privacy as handing out their device decryption keys on demand. Except with this new tech, finding criminal activity is even easier for law enforcement, as Apple will attempt to locate criminals for them. That said, there are better privacy tools available for those that need as much privacy as possible, but they are nowhere near as user-friendly as Apple's devices. For instance, if this new search tool only searches images uploaded to iCloud, as claimed, what stops a pedophile or a journalist from disabling iCloud photo storage and synchronizing photos manually? It is extra work, but not difficult. Keeping average everyday things and activities, such as web browsing history, mostly private requires more effort and expertise than manually copying photos between a phone and a computer.
- jasamer 5y ago
  The CSAM scanning solution allows law enforcement to do one thing, assuming they have full control over the image hash database: find people who have multiple photos that are already known to law enforcement that they upload to iCloud. How would you use this to find criminals in general? I have a hard time to think of a very useful example. Maybe photos of bomb plans downloaded somewhere from the internet that are known to be used by criminal groups? As to what stops a pedophile or a journalist form disabling iCloud photo storage: Absolutely nothing. This solution is supposed catch people storing CSAM photos in iCloud, that's it. A careful pedophile will not be caught by this.
shadowfacts 5y ago
While the precedent this sets is indeed concerning, the specific hypotheticals this article give are nonsensical. > an adversary could trick Apple’s algorithm into erroneously matching an existing image In which case the malicious, adversary-controlled images are sent to Apple. After which—the implication is—they can be re-obtained by... the adversary that created them. So what? An adversary could conceivably lower the reporting threshold by getting the victim to save a bunch of false-positive images. Again, so what? Surely if the adversary has reason to believe there are some number of CSAM images on a user's phone, there are more direct ways of going after them. > These kinds of false positives could happen if the matching database has been tampered with or expanded to include images that do not depict child abuse An adversary would either have to: A) carry out a supply-chain attack on Apple, B) ship different iOS images in different countries, or C) insert entries into the database on a specific users phone. Options A and C are irrelevant: if the phone is compromised, the CSAM database being modified is the least of your concerns. And option B is independently verifiable (granted, Apple does not do enough to make third-party auditing of iOS easy, but it is possible).
- kemayo 5y ago
  > An adversary would either have to: More importantly, they'd need to suborn Apple's human-review process, because the people doing the review would need to know what not-CSAM they're looking for. Could Apple be coerced into (or willingly) do this? I have no idea. But it's a very different threat-model than the article suggests, that boils down to the "do you trust your OS vendor?" question.
  - intricatedetail 5y ago
    How a human looking at low res CSAM matching collision picture that looks "innocent" will be able to tell for sure it is a false positive? Can they know with certainty that it is a false positive and not a manipulated real image? It seems to me that they would have to report these anyway.
    - kemayo 5y ago
deleted 5y ago
[deleted]
35803288 5y ago
This technology will soon be out of Apple’s control. Higgins correctly highlights the immense pressure Apple will get from governments and other actors to bend the technology and use it for something else than csam. It will happen, people are probably already thinking how to apply such pressure. Sooner or later Apple will cave in and they will have only themselves to blame when freedom supports in Sudan or LGBTQ activists in Saudi Arabia will be jailed. The issue here is not where to draw the line, but who will draw it (spoiler: not Apple).
- 35803288 5y ago
  Ten years from now we will read a leaked document stating that US authorities requested FISA Courts warrants (or similar) and made Apple scan for things other than csam. It already happened. Court orders are easier than hacks once you iron out the legal opinions.
  - sparker72678 5y ago
    Even if that's true, this method at least drops some part of the process on the client where it can be inspected, vs. all scanning happening in the cloud, entirely behind closed doors. Does that mean nothing bad can happen? No. But it does mean that when something changes, we at least know something changed.
    - jtbayly 5y ago
      The database will change every update. It’s not like the NCEMC db already has all possible CSAM. Every time they find more, they add it to the DB. Thus, it will probably change every time Apple pushes an OS update.
  - Lamad123 5y ago
    10 years seems a little too much for US gov's patience
- cyanite 5y ago
  Since it’s speculation at this point, I’d say we deal with it if and when it happens. I also don’t see how this doesn’t apply even worse to doing cloud side scanning, like companies otherwise do. Is that out of control? Is there any evidence of that?
sparker72678 5y ago
I'd love to be a privacy purist on this, but the fact is that there is not going to be any going back on CSAM content scanning of images in iCloud. So either we take this approach, or end up with a worse one, like pure on-cloud scanning with no transparency whatsoever. I read the technical paper today, and this solution is super clever, well considered, and checks just about every box a crypto-solution-phile would want. I don't know how it gets better than this. (Technical note: Apple's solution on this requires client and server communication to flag an image, which is part of the reason it's totally off if you turn off iCloud Photo). If this opens the door to Apple offering E2EE on all photos in iCloud (which it could), then this ends up being a big win over the status quo, in my opinion. And even if not, there are far, far worse alternatives. [Edited for spelling].
- almostdigital 5y ago
  > Apple's solution on this requires client and server communication to flag an image Well of course it does, would be a very pointless surveillance system if it scanned for stuff and then threw away the result. I've also read the technical papers and all other info put out by Apple so far and my takeaway is that the system is generic, it's not designed specifically for images but can be used with any old fingerprint and arbitrary data can be uploaded in the "security vouchers". It does not take much imagination to see how it could be used to mass-surveil E2EE communications, hook it into Siri's intent system, add a bunch of banned intents and throw the message into the voucher. If a threshold of bad intents is reached a reviewer will check your messages to verify that you actually had some bad intent before dispatching a patrol to your location.
  - sparker72678 5y ago
    Sure, you're totally right. But right now all that data is available to Apple. If they wanted to they could just generate a set of keys, replicate one of your iCloud devices, and send all your iMessages to a 3rd party, etc. If CSAM scanning is coming, show me a better solution. I don't think there's a world ahead of us that doesn't include it for all platform-hosted photos.
jug 5y ago
There’s been such a debate over this now that I don’t know why Apple are still trying? It’s not even their job to catch child porn. There are many far-reaching cooperations like at Interpol and Europol that successfully bust child porn groups? Even anonymizing networks like Tor are not safe havens. These guys don’t have it easy even today. So why is Apple suddenly extremely adamant about this? Would it even cost them anything to not go here? They’re a hardware and services company making mobile and computing devices mostly geared towards entertainment and creative work. How did they end up here?
- sparker72678 5y ago
  One possibility is that they'd like to go E2EE with all iCloud data, including photos, and this lets them do that without the politics (DC level) falling out against them.
  - heavyset_go 5y ago
    I doubt it, considering they give up customers' data when the government requests it for 150,000 users/accounts a year[1]. Not only do they choose to give data when requested, they have to comply with court orders, as well. Also, governments care about much more than just CSAM. They care about terrorism, drug manufacturing, drug and human trafficking, organized crime, gangs, fraud etc. This speculation only makes sense if Apple intends to use their CSAM detection and reporting system to detect and report on those other things, as well. Governments won't like that either, because during investigations and discovery, they'll want all of the customer's data that could be evidence of crimes. These detection and reporting systems, as implemented and by themselves, are only useful for flagging suspicious people. They're no good for complying with warrants or subpeonas. [1] https://www.apple.com/legal/transparency/us.html https://www.apple.com/legal/transparency/us.html
    - shuckles 5y ago
      I’ll take the other side of this and bet that Apple offers the ability to remove themselves from escrowing iCloud keys before they expand CSAM detection to other kinds of content. Nerd points are at stake.
BTCOG 5y ago
Fuck the cloud. Offer 256 and 512GB iPhones and up and give a feature set to entirely and permanently disable cloud function. Save that, and I'll throw my iPhone in the God damn trash. I'll use a raspberry Pi or flip phone if I have to do so.
- sparker72678 5y ago
  You can use an iPhone without ever logging into iCloud. You can even install MDM profiles to prevent it.
  - BTCOG 5y ago
    And I already do, but as a default standard I would just much rather see a phone large enough to never need iCloud but of course I didn't address the other issues that will now come from a proprietary phone. I will make good on buying a Librem 5 in the interim and throw the iPhone right in the dumpster though if they push forward with on device scanning and such. I never leave GPS turned on, I don't give any apps access to such features, and I buy phones outright in cash and put them on prepaid from the start. This is not good enough. If the next set of surveillance features are scanning messages, photos, videos on personal devices then it's nothing but a rented device and the only option are custom fully open source devices.
- 542458 5y ago
  Real talk, a Raspberry Pi phone could be amazing. A $150 (or similarly low cost) hackable phone with a near-guaranteed solid community around it - I’d buy that. One of my big disappointments is that canonical decided to go super-premium with their attempt at phones. I wish Firefox OS had done better (the f0x phone in particular was incredibly cool) but they never managed to get the sort of community traction they needed.
  - techdragon 5y ago
    Check out the PinePhone. While not exactly a “Raspberry Pi Phone” it’s pretty close to that in its basic concept.
JohnJamesRambo 5y ago
Apple has lost the nerds and I can’t think of a time that has ever gone well for a company. We are the people advising other people what to get, what is cool. I can’t think of anything less cool than an iPhone right now. I dumped mine and I don’t think I’m alone.
- deleted 5y ago
  [deleted]
- 99mans 5y ago
  Right now? We've known Apple is backdoored since at least 2013 with Snowden revelations. Some of us chose to pay attention, others seemingly went back to sleep ...
gerash 5y ago
If someone is ok with Apple handing over iCloud keys to the Chinese government to comply with local laws then scanning for CSAM is no different. Basically, if you have issue with this go change the law rather than ask large corporations to act illegally for some greater good.

There business model has always been a threat to user privacy and press freedom, many people were just okay with it cause Apple told them otherwise. Open source is the only model where you can verify though, and so Apple was about blind trust and never about privacy.

99mans 5y ago
Why is this being downvoted? Depressing to see that open source software is not respected here.
- sandworm101 5y ago
  People downvote things that make them angry. The things that make people most angry are those things that strike closest to home. An incorrect or unacceptable comment will be downvoted and commented on, even reported. A perfectly valid and true statement, one that is so true that it scares people, will be silently downvoted. It is a form of denial, an attempt to remove a painful truth.
- simondotau 5y ago
  I didn't downvote it, but I can understand why some people might have. Because while it's superficially true, it's disingenuous in practice. All security is, at its core, a network of trust between you and other entities. Open source isn't secure because it's open, it's secure because you trust Canonical, you trust Linus Torvalds, you trust GNU, etc. And those people trust other people—hence the "network" of trust. Saying that you can inspect the source code is true in theory. But unless you've done the full audit yourself, you're personally as blind as you are with closed source code. You're choosing to trust whichever security researchers deeply understand the security implications of all the gobbledegook in all the USB drivers, and you're trusting that Canonical is shipping you the same version that security researchers have validated. For 99.9% of users, it all comes down to blind trust. As Linus himself once said: "If you have ever done any security work and it didn't involve the concept of a network of trust, it was not a security work. It was masturbation."
npteljes 5y ago
I agree, this is what's to be expected when people rely on proprietary and other people's computers. Stallman's essays, among others, depicted scenarios like this for ages. But people don't like to miss the latest shiny, the convenience of the smart phone tech, so, here we are. They also don't like a told-you-so.

deleted 5y ago

[deleted]

fmajid 5y ago

Of course. Politicians and officials hate whistleblowers far more than pedophiles, and it's obvious this mechanism will be used to find future Chelsea Mannings and Reality Winners with better opsec to make examples out of them.

HNPoliticalBias 5y ago
Any population that allows themselves to be ruled by tyrants, will be, as we're seeing with the West slowly morphing into the USSR.
sparker72678 5y ago
How? I'd love to be convinced that this mechanism could be easily extended to any content at any time without it being totally obvious to the rest of us that it's being used that way.
- jjcon 5y ago
  Not OP and not speaking with any authority on this more just curious: If alongside this Apple rolled out the ability for them to initiate a scan on any phone for any thing (not a tool for mass surveillance but targeted surveillance) how would we know given Apples closed ecosystem? This doesn't necessarily speak to the issue here but more to the Apple ethos in general, but I am curious.
  - simondotau 5y ago
    This twitter thread is a useful read: https://twitter.com/pwnallthethings/status/1424873629003702273 https://twitter.com/pwnallthethings/status/14248736290037022...
    - techdragon 5y ago
      Very good breakdown of the situation. But cold comfort for anyone not in the USA. Because we know once Apple ships this it will be expanded to other countries with similar CSAM laws … many of which do not have robust Fourth Amendment type rights enshrined in their legal systems.

threeseed 5y ago

This article is full of inaccuracies. a) People have extracted a pre-release version from an older iOS phone. It is not the one that will be released and not the same one Apple uses server-side to verify the process. b) Adversaries can not reasonably break this process by flooding it with bad data across a range of compromised phones. The client side version is there to prevent this as is the fact it would require jailbroken devices with iCloud logins. c) If you can't trust Apple not to include non-CSAM hashes in the database then how can you trust them not to compromise the operating system itself ? It's illogical. Apple verifies hashes against two sources and two manual checks are required thus needing multiple failure points in the process.

mattnewton 5y ago
For b), I can't see how creating spurious icloud accounts and spoofing it will be hard at it's face. I have made dozens of icloud accounts personally for testing in the past. They may take steps to address this attack vector, it's not an unsolvable problem but it isn't solved by requiring a device or icloud account as far as I can tell. For c), the problem isn't just trusting Apple, for whom the hashes are somewhat opaque, my understanding is that they are provided by a governmentally regulated third party.
- kemayo 5y ago
  For c) you still need to imagine a scenario where Apple is deliberately complicit, because of the human review step. If non-CSAM hashes are inserted into the database, the human reviewers need to know what non-CSAM they're looking for -- otherwise they'll see the e.g. picture of a leaked document, observe that it's not child porn, and flag it as a false positive.
  - intricatedetail 5y ago
    How would they know it is a false positive and not a CSAM image concealed in a document image that NeuralHash was able to "see"? It will be impossible to tell from a lowres picture with certainty.

mrharrison 5y ago

There is an easy way to cast your vote for saying yes to Privacy. Turn off auto-updates and don't update to iOS 15. Spread the word.

hypothesis 5y ago
Presumably, there will be a parallel track to stay on iOS 14 with security updates, at least for some time.
- SSLy 5y ago
  This tech is in iOS since 14.3
  - hypothesis 5y ago
    The neuralHash part, not the whole CSAM scanning daemon/pipeline, right? Security updates to iOS 14 will let people (who might not be ready/able to switch from iOS) to actually do what GP said, without unnecessarily exposing themselves to security bugs in outdated OS.
- OneLeggedCat 5y ago
  If enough people do that, then no reason that they won't just enable it in 14 too.
  - hypothesis 5y ago
    Assuming that I was not gaslit by other people, I was under impression that would at least require some change to EULA. Otherwise it sounds like a nuclear option by Apple, with dire effects.
    - greyface- 5y ago
      iOS 14 EULA: > By using the Apple Software, you agree that Apple may download and install automatic Apple Software Updates onto your Device and your peripheral devices. > Apple and its licensors reserve the right to change, suspend, remove, or disable access to any Services at any time without notice.

jeremygaither 5y ago

I agree with the article that not only are implementation problems a potential threat, but it also sets a dangerous precedent. I also understand that Apple is attempting to compromise with governments that are still pressuring it to create an encryption back door or hand over keys (which it did in China by having another company operate iCloud there). Governments have often used the pursuit and prosecution of child abusers and terrorists as a scapegoat for eroding privacy for all its citizens. Privacy and liberty versus the lawful pursuit of criminals are challenging problems to balance, especially when governments and law enforcement want a free pass searching for whatever content they deem illegal. Governments decide what photos, communications, and even thoughts (if they could) are legal and which one are illegal. Who knows what some (new) governments may decree new obscenity laws? They could legislate that some pictures once considered lawful and innocent are now unlawful and obscene, retroactively, and ask or attempt to force Apple to search for those newly criminal images. Apple’s new image scanning technology is almost as much of a backdoor to individuals' privacy as handing out their device decryption keys on demand. Except with this new tech, finding criminal activity is even easier for law enforcement, as Apple will attempt to locate criminals for them. That said, there are better privacy tools available for those that need as much privacy as possible, but they are nowhere near as user-friendly as Apple's devices. For instance, if this new search tool only searches images uploaded to iCloud, as claimed, what stops a pedophile or a journalist from disabling iCloud photo storage and synchronizing photos manually? It is extra work, but not difficult. Keeping average everyday things and activities, such as web browsing history, mostly private requires more effort and expertise than manually copying photos between a phone and a computer.

jasamer 5y ago
The CSAM scanning solution allows law enforcement to do one thing, assuming they have full control over the image hash database: find people who have multiple photos that are already known to law enforcement that they upload to iCloud. How would you use this to find criminals in general? I have a hard time to think of a very useful example. Maybe photos of bomb plans downloaded somewhere from the internet that are known to be used by criminal groups? As to what stops a pedophile or a journalist form disabling iCloud photo storage: Absolutely nothing. This solution is supposed catch people storing CSAM photos in iCloud, that's it. A careful pedophile will not be caught by this.

shadowfacts 5y ago

While the precedent this sets is indeed concerning, the specific hypotheticals this article give are nonsensical. > an adversary could trick Apple’s algorithm into erroneously matching an existing image In which case the malicious, adversary-controlled images are sent to Apple. After which—the implication is—they can be re-obtained by... the adversary that created them. So what? An adversary could conceivably lower the reporting threshold by getting the victim to save a bunch of false-positive images. Again, so what? Surely if the adversary has reason to believe there are some number of CSAM images on a user's phone, there are more direct ways of going after them. > These kinds of false positives could happen if the matching database has been tampered with or expanded to include images that do not depict child abuse An adversary would either have to: A) carry out a supply-chain attack on Apple, B) ship different iOS images in different countries, or C) insert entries into the database on a specific users phone. Options A and C are irrelevant: if the phone is compromised, the CSAM database being modified is the least of your concerns. And option B is independently verifiable (granted, Apple does not do enough to make third-party auditing of iOS easy, but it is possible).

kemayo 5y ago
> An adversary would either have to: More importantly, they'd need to suborn Apple's human-review process, because the people doing the review would need to know what not-CSAM they're looking for. Could Apple be coerced into (or willingly) do this? I have no idea. But it's a very different threat-model than the article suggests, that boils down to the "do you trust your OS vendor?" question.
- intricatedetail 5y ago
  How a human looking at low res CSAM matching collision picture that looks "innocent" will be able to tell for sure it is a false positive? Can they know with certainty that it is a false positive and not a manipulated real image? It seems to me that they would have to report these anyway.
  - kemayo 5y ago

deleted 5y ago

[deleted]

35803288 5y ago

This technology will soon be out of Apple’s control. Higgins correctly highlights the immense pressure Apple will get from governments and other actors to bend the technology and use it for something else than csam. It will happen, people are probably already thinking how to apply such pressure. Sooner or later Apple will cave in and they will have only themselves to blame when freedom supports in Sudan or LGBTQ activists in Saudi Arabia will be jailed. The issue here is not where to draw the line, but who will draw it (spoiler: not Apple).

35803288 5y ago
Ten years from now we will read a leaked document stating that US authorities requested FISA Courts warrants (or similar) and made Apple scan for things other than csam. It already happened. Court orders are easier than hacks once you iron out the legal opinions.
- sparker72678 5y ago
  Even if that's true, this method at least drops some part of the process on the client where it can be inspected, vs. all scanning happening in the cloud, entirely behind closed doors. Does that mean nothing bad can happen? No. But it does mean that when something changes, we at least know something changed.
  - jtbayly 5y ago
    The database will change every update. It’s not like the NCEMC db already has all possible CSAM. Every time they find more, they add it to the DB. Thus, it will probably change every time Apple pushes an OS update.
- Lamad123 5y ago
  10 years seems a little too much for US gov's patience
cyanite 5y ago
Since it’s speculation at this point, I’d say we deal with it if and when it happens. I also don’t see how this doesn’t apply even worse to doing cloud side scanning, like companies otherwise do. Is that out of control? Is there any evidence of that?

sparker72678 5y ago

I'd love to be a privacy purist on this, but the fact is that there is not going to be any going back on CSAM content scanning of images in iCloud. So either we take this approach, or end up with a worse one, like pure on-cloud scanning with no transparency whatsoever. I read the technical paper today, and this solution is super clever, well considered, and checks just about every box a crypto-solution-phile would want. I don't know how it gets better than this. (Technical note: Apple's solution on this requires client and server communication to flag an image, which is part of the reason it's totally off if you turn off iCloud Photo). If this opens the door to Apple offering E2EE on all photos in iCloud (which it could), then this ends up being a big win over the status quo, in my opinion. And even if not, there are far, far worse alternatives. [Edited for spelling].

almostdigital 5y ago
> Apple's solution on this requires client and server communication to flag an image Well of course it does, would be a very pointless surveillance system if it scanned for stuff and then threw away the result. I've also read the technical papers and all other info put out by Apple so far and my takeaway is that the system is generic, it's not designed specifically for images but can be used with any old fingerprint and arbitrary data can be uploaded in the "security vouchers". It does not take much imagination to see how it could be used to mass-surveil E2EE communications, hook it into Siri's intent system, add a bunch of banned intents and throw the message into the voucher. If a threshold of bad intents is reached a reviewer will check your messages to verify that you actually had some bad intent before dispatching a patrol to your location.
- sparker72678 5y ago
  Sure, you're totally right. But right now all that data is available to Apple. If they wanted to they could just generate a set of keys, replicate one of your iCloud devices, and send all your iMessages to a 3rd party, etc. If CSAM scanning is coming, show me a better solution. I don't think there's a world ahead of us that doesn't include it for all platform-hosted photos.

jug 5y ago

There’s been such a debate over this now that I don’t know why Apple are still trying? It’s not even their job to catch child porn. There are many far-reaching cooperations like at Interpol and Europol that successfully bust child porn groups? Even anonymizing networks like Tor are not safe havens. These guys don’t have it easy even today. So why is Apple suddenly extremely adamant about this? Would it even cost them anything to not go here? They’re a hardware and services company making mobile and computing devices mostly geared towards entertainment and creative work. How did they end up here?

sparker72678 5y ago
One possibility is that they'd like to go E2EE with all iCloud data, including photos, and this lets them do that without the politics (DC level) falling out against them.
- heavyset_go 5y ago
  I doubt it, considering they give up customers' data when the government requests it for 150,000 users/accounts a year[1]. Not only do they choose to give data when requested, they have to comply with court orders, as well. Also, governments care about much more than just CSAM. They care about terrorism, drug manufacturing, drug and human trafficking, organized crime, gangs, fraud etc. This speculation only makes sense if Apple intends to use their CSAM detection and reporting system to detect and report on those other things, as well. Governments won't like that either, because during investigations and discovery, they'll want all of the customer's data that could be evidence of crimes. These detection and reporting systems, as implemented and by themselves, are only useful for flagging suspicious people. They're no good for complying with warrants or subpeonas. [1] https://www.apple.com/legal/transparency/us.html https://www.apple.com/legal/transparency/us.html
  - shuckles 5y ago
    I’ll take the other side of this and bet that Apple offers the ability to remove themselves from escrowing iCloud keys before they expand CSAM detection to other kinds of content. Nerd points are at stake.

BTCOG 5y ago

Fuck the cloud. Offer 256 and 512GB iPhones and up and give a feature set to entirely and permanently disable cloud function. Save that, and I'll throw my iPhone in the God damn trash. I'll use a raspberry Pi or flip phone if I have to do so.

sparker72678 5y ago
You can use an iPhone without ever logging into iCloud. You can even install MDM profiles to prevent it.
- BTCOG 5y ago
  And I already do, but as a default standard I would just much rather see a phone large enough to never need iCloud but of course I didn't address the other issues that will now come from a proprietary phone. I will make good on buying a Librem 5 in the interim and throw the iPhone right in the dumpster though if they push forward with on device scanning and such. I never leave GPS turned on, I don't give any apps access to such features, and I buy phones outright in cash and put them on prepaid from the start. This is not good enough. If the next set of surveillance features are scanning messages, photos, videos on personal devices then it's nothing but a rented device and the only option are custom fully open source devices.
542458 5y ago
Real talk, a Raspberry Pi phone could be amazing. A $150 (or similarly low cost) hackable phone with a near-guaranteed solid community around it - I’d buy that. One of my big disappointments is that canonical decided to go super-premium with their attempt at phones. I wish Firefox OS had done better (the f0x phone in particular was incredibly cool) but they never managed to get the sort of community traction they needed.
- techdragon 5y ago
  Check out the PinePhone. While not exactly a “Raspberry Pi Phone” it’s pretty close to that in its basic concept.

JohnJamesRambo 5y ago

Apple has lost the nerds and I can’t think of a time that has ever gone well for a company. We are the people advising other people what to get, what is cool. I can’t think of anything less cool than an iPhone right now. I dumped mine and I don’t think I’m alone.

deleted 5y ago
[deleted]
99mans 5y ago
Right now? We've known Apple is backdoored since at least 2013 with Snowden revelations. Some of us chose to pay attention, others seemingly went back to sleep ...

gerash 5y ago

If someone is ok with Apple handing over iCloud keys to the Chinese government to comply with local laws then scanning for CSAM is no different. Basically, if you have issue with this go change the law rather than ask large corporations to act illegally for some greater good.