8 ms·

>So what happens when, in a few years at the latest, a politician points that out, and—in order to protect the children—bills are passed in the legislature to p

by slg 5y ago

>So what happens when, in a few years at the latest, a politician points that out, and—in order to protect the children—bills are passed in the legislature to prohibit this "Disable" bypass, effectively compelling Apple to scan photos that aren’t backed up to iCloud? What happens when a party in India demands they start scanning for memes associated with a separatist movement? What happens when the UK demands they scan for a library of terrorist imagery? How long do we have left before the iPhone in your pocket begins quietly filing reports about encountering “extremist” political material, or about your presence at a "civil disturbance"? Or simply about your iPhone's possession of a video clip that contains, or maybe-or-maybe-not contains, a blurry image of a passer-by who resembles, according to an algorithm, "a person of interest"? What I don't get is what prevented these things from happening last month? Apple controls the hardware, the software, and the cloud services so the point at which the scanning is done is mostly arbitrary from a process standpoint (I understand people believe there are huge differences philosophically). They could have already scanned our files because they already have full control over the entire ecosystem. If they can be corrupted by authoritative governments, then shouldn't we assume that have already been corrupted? If so, why did we trust them with full control of the ecosystem?

barsonme 5y ago
Because that door hasn’t been opened yet. “Scan every photo on users devices” or “scan for non-CSAM” are much easier requests once they’ve already started scanning on-device. It’s just how life and politics work.
- anonuser123456 5y ago
  The door has been opened for quite some time. What do you think spotlight is? It scans an indexes all your data. What's prevented the government from saying "hey if you see Osama Bin Laden in a spotlight scan, you need to send us all that guys data." The answer is, Apple can just say FU. And that's exactly what will happen here. In particular, the US DOJ needs to stay in Apple's good graces here and not be overly aggressive. If DOJ pulls any funny business, that's a pretty good reason for Apple to just say "OK, we're picking up our toys and going home. You get nothing now and we're turning on E2EE."
  - barsonme 5y ago
    I’m sorry, but there is a world of difference between locally indexing files for local search and tagging files as contraband so that they can be reported to the government.
    - naravara 5y ago
      Technically speaking, no there isn’t. It’s just a little bit if metadata you’re sticking on an already long tail of metadata that they’re doing as they index.
      - shuckles 5y ago
        Most importantly, people are arguing that there is new technical risk, so a difference in intent or beneficiaries is not relevant for that argument.
      - barsonme 5y ago
        Being technically similar is irrelevant. It’s completely different in principle, and that’s what matters. A great deal many things are technically similar, but vastly different in principle. And those distinctions are important.
        snowwrestler 5y ago
        I’m not sure which side you’re arguing here. The biggest concern about Apple’s system is that it’s very easy to add new items to a hash list. That is an argument about the technical similarity of scanning for CSAM and scanning for other things like classified documents (for example). But there is a vast difference in principle. Pretty much everyone wants to stop child abuse. But many people—including major news organizations—believe citizens should sometimes have the opportunity to view classified documents. Different categories of things to scan for will be different in principle, even if the technical approach is similar. This difference in principle is what Apple leans on when they say they will oppose any request to expand their system beyond CSAM.
kevin_thibedeau 5y ago
It isn't a philosophical debate. It's about invading and controlling someone else's property. I can't shack up in your home and eat your food just because I feel like it. We're all doomed because digital natives have no concept of boundaries between something they own and something someone is renting or letting you use for free in exchange for data mining.
- codetrotter 5y ago
  I agree we are all doomed, but I don’t agree it has that much with digital native or not to do. My boomer grandparents, my gen x parents and my millennial self, we are all affected by this. And gen z (the first generation of digital natives), and whatever comes after gen z, is not to blame for that. Reducing it to a generational thing is silly.
  - tartoran 5y ago
    I think the point was that the digital natives and the next generation of digital natives coming will not know any different and will thus tacitly accept it.
- judge2020 5y ago
  Apple is renting the phone to you for $1000 down and $0 a month (unless you actually are financing). Therefore, they are the landlord and, given notice, can change the property as they feel fit.
  - least 5y ago
    This is demonstrably not true. If you rent a home and then burn it down, you are going to be held liable to the owner of the home. In the case of your phone, no one, including Apple, cares if you buy it and then immediately smash it on the ground and destroy it. Apple controls the software that runs on it but there is nothing that stops you from modifying or hacking it to your heart's content if you are able to, just as they are not obligated to make that an easy task for you.
polishdude20 5y ago
At one point this will be proven and we'll go back to regular digital cameras or even polaroids.
lvs 5y ago
That's rational, but the point he's making is that this system obliterates the only defense we have had or could have against such activity: end-to-end encryption. This approach owns the endpoint.
- shuckles 5y ago
  …in the same way any existing feature of iOS that makes device data available to Apple (eg iCloud Backup) “owns” the endpoint, no? What’s to stop a malicious Apple from turning on iCloud Backup for all its users and hoovering up your Signal messages database and iCloud Keychain?
  - sneak 5y ago
    Nothing. iOS even defaults autoupdate to on, so Apple could do this without your interaction today.
diebeforei485 5y ago
There is a fairly large difference, first being it would be a massive damage to Apple's brand if they started scanning people's phones without permission. But now that they've built the system to scan things on-device, they can be compelled by a government to scan for other things, and Apple can shrug their hands and say they had no choice.
- lupire 5y ago
  Why would Apple start shrugging now when they've been fighting the FBI in court?
  - diebeforei485 5y ago
    One reason is that they weren't under antitrust scrutiny in 2016 when they fought the the government in court. Their incentives have changed - they now have the real looming threat of being broken up by governments, so it is now in their interest to comply with anything else governments ask them to do.
    - anshumankmr 5y ago
      How do you think will the lawyers be able to prove Apple is a monopoly when Android exists?
      - diebeforei485 5y ago
        The mere existence of a competitor doesn't matter in competition law. Other oil companies existed, but the government still broke up Standard Oil. Other browsers existed, but the government still made Microsoft make certain API's available to other browser makers.
        anshumankmr 5y ago
        This is a false equivalence. Standard Oil was at the top. Apple is clearly not. In the USA, they may very well be, but I doubt the margin is very high. https://images.app.goo.gl/hLr1zAGs7Wq5TG46A https://images.app.goo.gl/hLr1zAGs7Wq5TG46A A lawyer will say that Apple though in the lead as a brand, is still second to Android in terms of market share, no matter how fragmented the market is. This might be the same in other western countries as well. But overall Android would still have more devices running it than iOS.
swiley 5y ago
Nothing except Apple saying you could trust them. People were stupid enough to accept that and now even the trust is gone.
fshbbdssbbgdd 5y ago
I think “we don’t have the machinery to do that” is an effective argument in the real world when someone asks you do to something. I’m not sure if it matters legally (lawyers sometimes use vague phrases like “reasonable effort”), but it definitely affects how strongly people will pressure you to do things, and how likely you are to acquiesce to that pressure. The scope of the change Apple would need to make to scan your photos arbitrarily just got a lot smaller. The number of engineers who would need to be “in the know” to implement this change got smaller. The belief from governments that Apple has the option of doing this got stronger. The belief among Apple’s own management team that they can do this got stronger.
- snowwrestler 5y ago
  This is very well put.
croutonwagon 5y ago
In years previous, take the San Bernadino shooter for instance, Apple argued in the court of law that creating backdoors or reversible encryption was insecure and also subject to exploits by malicious actors, and thus not reasonable and was "unreasonably burdensome". As well, they made the argument that compelling them to do write back doors also violated the first amendment. It was most likely a winning strategy that the FBI actively avoided getting rulings on and found a workaround. What apple is creating here is an avenue for the FBI/NSA/Alphabet agency to create a FISA warrant and NDL to mandate hits on anything. The argument its gotta be pre-icloud upload or subject to manual review or on some arbitrary threshold is something is just the marketing to get the public to accept it. All of that can easily be ordered to be bypassed. So it can be a scan, single hit for x, report. Ill take the downvotes, but if anything, someone more conspiracy minded could easily take this as a warrant canary. Given the backlash apple ahs faced and ignored, it doesnt make much good business sense for them not to back off unless they are A) betting on it being a vocal minority to resorts to action (which is entirely possible, especially given the alternatives and technical hurdles to get to a suitable alternative) B) Being pressured by governments now. (also entirely possible given their history with the FBI and previous investigations). [1] https://www.rpc.senate.gov/policy-papers/apple-and-the-san-bernardino-terror-attack https://www.rpc.senate.gov/policy-papers/apple-and-the-san-b... [2] https://en.wikipedia.org/wiki/FBI%E2%80%93Apple_encryption_dispute#Apple's_opposition_to_the_order https://en.wikipedia.org/wiki/FBI%E2%80%93Apple_encryption_d...
- nicce 5y ago
  > What apple is creating here is an avenue for the FBI/NSA/Alphabet agency to create a FISA warrant and NDL to mandate hits on anything. The argument its gotta be pre-icloud upload or subject to manual review or on some arbitrary threshold is something is just the marketing to get the public to accept it. Why would they make things even more complicated with limited access, since they can already access everything in cloud? Let’s leave out the argument for expanding scan to whole device. If that is what happens, then people start really discarding their phones.
  - klipklop 5y ago
int_19h 5y ago
The politics of it is very different, and that's where the danger lies: https://news.ycombinator.com/item?id=28239506 https://news.ycombinator.com/item?id=28239506 I think that quite a few engineers are too focused on the technical aspects of it, and specifically on all those "barriers to misuse" that Apple claims to have in place. But it'll be much easier to remove the barriers once the system as a whole is in place.
- judge2020 5y ago
  The reason we're focused on the state of it now is that we can switch at any time - especially if those barriers are shown to be ineffective or are removed at some point.
  - int_19h 5y ago
    The real threat here is legal, not technical. Think mandatory on-device scanning as condition of access to the hardware market.
    - judge2020 5y ago
      That just ties back into "be afraid of what it could become", and isn't dependent on Apple making this system - Congress could have forced PhotoDNA to be shipped with phones since the inception of PhotoDNA in 2011[0]. 0: https://doi.org/10.1016/S0262-4079(11)60791-4 https://doi.org/10.1016/S0262-4079(11)60791-4
      - int_19h 5y ago
        Of course it's not dependent on it being Apple. It could just as well have been Google. And the difference is that Apple gave them what they needed to mandate this while still claiming that they preserve privacy.
sillystuff 5y ago
> What I don't get is what prevented these things from happening last month? Apple controls the hardware, the software, and the cloud services... Yes, proprietary black-box hardware and software is poor from a user privacy perspective. But, If Apple began on-device scanning of content, I'd imagine eventually someone would notice the suspicious activity and investigate. With Apple's announcement, the scanning will just be something that Apple devices do. Nothing to worry about. And, no way for anyone to independently verify that the scope of the content being scanned has not been secretly increased. As for icloud, if your content is not encrypted on the device in a manner where only you have the keys, any cloud storage is suspect for scanning / data mining. But, on-device scanning is a back door for e2e encryption-- even on device encryption with keys only you control is thwarted.
- shuckles 5y ago
  How do you think Apple will increase the scope of what’s scanned without every person with Ghidra skills not noticing?
  - sillystuff 5y ago
    If the exchange with Apple is encrypted / interleaved with other traffic to icloud, how would you know that there isn't new classes of scanning being done? I'll be very surprised if similar tech is not lobbied for as a backstop to catch DRM-free media files played on devices we "own". And, it seems far more probable than not that police will demand this capability be used to help address more crimes. The problem here is that crimes can mean speaking out against an oppressive regime. Being targeted for having the wrong political views (think McCarthyism in the United States or the US backed murder of a million people in Indonesia for affiliating with the "wrong" political party). Etc. History is awash with political abuse of "out groups" perpetrated by tin pot dictators all the way to presidents and PMs of major world powers. And, it sets the precedent that e2e encryption is not an excuse for a provider to not provide private customer data to the authorities-- a back-door can be installed, "Just do what Apple did."
- JimDabell 5y ago
  > no way for anyone to independently verify that the scope of the content being scanned has not been secretly increased. This seems like the easiest thing out of the lot to verify. The way that this system is designed to work is that when uploading to iCloud Photos, images have a safety voucher attached to them. If Apple secretly expanded this to scan more than just iCloud Photos, they would have to either a) upload all the extra photos, b) add a new mechanism to upload just the vouchers, or c) upload “fake” photos to iCloud Photos with the extra vouchers attached. None of these seem particularly easy to disguise. Your concern is completely understandable if you are starting from the premise that Apple are scanning photos then uploading matches. I think that’s how a lot of people are assuming this works, but that’s not correct. Apple designed the system in a very different way that is integrated into the iCloud upload process, and that design makes it difficult to expand the scope beyond iCloud Photos surreptitiously. Could Apple build a system to secretly exfiltrate information from your phone? Of course. They could have done so since the first iPhone was released in 2007. But this design that they are actually using is an awful design if that’s what they wanted to do. All of their efforts on this seem to be pointed in the exact opposite direction.
NicoJuicy 5y ago
> They could have already scanned our files because they already have full control over the entire ecosystem They did do it in emails since 2019: https://www.indiatoday.in/technology/news/story/apple-has-been-scanning-icloud-mail-for-child-exploitation-images-since-2019-1844732-2021-08-24 https://www.indiatoday.in/technology/news/story/apple-has-be...
noptd 5y ago
This entire argument is a non sequitur and comes up like clockwork every time this issue is discussed. It's the metaphorical equivalent of saying "well someone could've snuck in through the open window. Let's just assume they did and leave the doors open as well". How about instead we push back against Apple further shifting the Overton window on how acceptable it is for companies to run intrusive services on hardware we own?
- shuckles 5y ago
  It’s not a non sequitur. The comment is engaging with a series of rhetorical questions that imagine a slippery slope by observing that very little has changed about the trust model between iPhone users and their devices. If you are convinced Apple is slipping, then it is worthwhile to be able to answer how their position today is different than it was last month. That is of course a different question than whether their position last month was acceptable, and maybe people are realizing it was not. As a concrete example, if you think the proposal introduces new technical risks, then if Apple announces they made a mistake and will instead scan entirely on the server, you may be satisfied. However, I’d argue that since no new technical risk has been introduced, your conclusions should not change. I’d argue that the incorrect characterization of Apple’s announcement as scanning all the files on your phone with no control has shifted the Overton window more than what was actually proposed. Politicians who are none the wiser probably believe that’s what Apple actually built, even though it’s not.
  - noptd 5y ago
    I disagree - it's a distraction from the larger issue at hand. > I’d argue that the incorrect characterization of Apple’s announcement as scanning all the files on your phone with no control That's a strawman - few if any are arguing that the system will read all of your files out of the gate. >since no new technical risk has been introduced, This assumption doesn't reflect reality. Introducing a brand new system built specifically for client-side scanning absolutely adds technical risks, if nothing else then by the sheer fact that it's adding another attack vector on your phone. Not to mention the fact that all it would take is a change in policy and a few trivial updates (a new event trigger, directory configs, etc) for this system to indeed scan any file on your device.
rkagerer 5y ago
What I don't get is what prevented these things from happening last month? Apple controls the hardware, the software, and the cloud services... Simple: Money. Their response to any such demands would be (and has been) "we don't have the capability to do what you're asking". No judge is going to burden them to spend their own dime to build a massive new feature like this and deploy it to every phone out there to comply with a demand arising from a prosecutor of an individual case. No government is going to pony up the money to reimburse them to do it (not even getting into the PR optics). That leaves it happening only if 1) they decide to do it themselves, or 2) government(s) legislate they must. So far #2 hasn't happened. Politicians had no basis of reference to point to and say "Your competitor(s)' doing that, you should too". But now that #1 occurred, it will normalize this nonsense and pave the way for #2.
- WillPostForFood 5y ago
  Their response to any such demands would be (and has been) "we don't have the capability to do what you're asking". No judge is going to burden them to spend their own dime to build a massive new feature like this and deploy it to every phone out there to comply with a demand arising from a prosecutor of an individual case. Government does not care one bit about how much it costs or if it is even possible. They demand the data with an ultimatum: deliver it as we requested by our deadline or we send in our IT people to take it. Sorry (not) if it takes your whole company down while we plugin in our own servers in your datacenter to take your data.
  - rkagerer 5y ago
    Doesn't work if the data of interest is not there for the taking. And a judge will not compel beyond what they consider reasonable. Having the feature already in place dramatically shifts the bar.
- shuckles 5y ago
  Their response to such demands has not been we are technically incapable of doing what’s requested. The demand from the FBI in the San Bernardino case was a very small change to passcode retry constants, because the terrorist’s device did not have a Secure Element.
judge2020 5y ago
> They could have already scanned our files because they already have full control over the entire ecosystem. Apple barely submits any CSAM[0]: > According to NCMEC, I submitted 608 reports to NCMEC in 2019, and 523 reports in 2020. In those same years, Apple submitted 205 and 265 reports (respectively). It isn't that Apple doesn't receive more picture than my service, or that they don't have more CP than I receive. Rather, it's that they don't seem to notice and therefore, don't report. 0: https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On...
xunn0026 5y ago
The existing law and user agreement also sets your rights. Apple might have done what you say last month, illegally. Now it's in the user agreement and they can do it legally, at scale. This literally creates a precedent.

by slg 5y ago

barsonme 5y ago

Because that door hasn’t been opened yet. “Scan every photo on users devices” or “scan for non-CSAM” are much easier requests once they’ve already started scanning on-device. It’s just how life and politics work.

anonuser123456 5y ago
The door has been opened for quite some time. What do you think spotlight is? It scans an indexes all your data. What's prevented the government from saying "hey if you see Osama Bin Laden in a spotlight scan, you need to send us all that guys data." The answer is, Apple can just say FU. And that's exactly what will happen here. In particular, the US DOJ needs to stay in Apple's good graces here and not be overly aggressive. If DOJ pulls any funny business, that's a pretty good reason for Apple to just say "OK, we're picking up our toys and going home. You get nothing now and we're turning on E2EE."
- barsonme 5y ago
  I’m sorry, but there is a world of difference between locally indexing files for local search and tagging files as contraband so that they can be reported to the government.
  - naravara 5y ago
    Technically speaking, no there isn’t. It’s just a little bit if metadata you’re sticking on an already long tail of metadata that they’re doing as they index.
    - shuckles 5y ago
      Most importantly, people are arguing that there is new technical risk, so a difference in intent or beneficiaries is not relevant for that argument.
    - barsonme 5y ago
      Being technically similar is irrelevant. It’s completely different in principle, and that’s what matters. A great deal many things are technically similar, but vastly different in principle. And those distinctions are important.
      - snowwrestler 5y ago
        I’m not sure which side you’re arguing here. The biggest concern about Apple’s system is that it’s very easy to add new items to a hash list. That is an argument about the technical similarity of scanning for CSAM and scanning for other things like classified documents (for example). But there is a vast difference in principle. Pretty much everyone wants to stop child abuse. But many people—including major news organizations—believe citizens should sometimes have the opportunity to view classified documents. Different categories of things to scan for will be different in principle, even if the technical approach is similar. This difference in principle is what Apple leans on when they say they will oppose any request to expand their system beyond CSAM.

kevin_thibedeau 5y ago

It isn't a philosophical debate. It's about invading and controlling someone else's property. I can't shack up in your home and eat your food just because I feel like it. We're all doomed because digital natives have no concept of boundaries between something they own and something someone is renting or letting you use for free in exchange for data mining.

codetrotter 5y ago
I agree we are all doomed, but I don’t agree it has that much with digital native or not to do. My boomer grandparents, my gen x parents and my millennial self, we are all affected by this. And gen z (the first generation of digital natives), and whatever comes after gen z, is not to blame for that. Reducing it to a generational thing is silly.
- tartoran 5y ago
  I think the point was that the digital natives and the next generation of digital natives coming will not know any different and will thus tacitly accept it.
judge2020 5y ago
Apple is renting the phone to you for $1000 down and $0 a month (unless you actually are financing). Therefore, they are the landlord and, given notice, can change the property as they feel fit.
- least 5y ago
  This is demonstrably not true. If you rent a home and then burn it down, you are going to be held liable to the owner of the home. In the case of your phone, no one, including Apple, cares if you buy it and then immediately smash it on the ground and destroy it. Apple controls the software that runs on it but there is nothing that stops you from modifying or hacking it to your heart's content if you are able to, just as they are not obligated to make that an easy task for you.

polishdude20 5y ago

At one point this will be proven and we'll go back to regular digital cameras or even polaroids.

lvs 5y ago

That's rational, but the point he's making is that this system obliterates the only defense we have had or could have against such activity: end-to-end encryption. This approach owns the endpoint.

shuckles 5y ago
…in the same way any existing feature of iOS that makes device data available to Apple (eg iCloud Backup) “owns” the endpoint, no? What’s to stop a malicious Apple from turning on iCloud Backup for all its users and hoovering up your Signal messages database and iCloud Keychain?
- sneak 5y ago
  Nothing. iOS even defaults autoupdate to on, so Apple could do this without your interaction today.

diebeforei485 5y ago

There is a fairly large difference, first being it would be a massive damage to Apple's brand if they started scanning people's phones without permission. But now that they've built the system to scan things on-device, they can be compelled by a government to scan for other things, and Apple can shrug their hands and say they had no choice.

lupire 5y ago
Why would Apple start shrugging now when they've been fighting the FBI in court?
- diebeforei485 5y ago
  One reason is that they weren't under antitrust scrutiny in 2016 when they fought the the government in court. Their incentives have changed - they now have the real looming threat of being broken up by governments, so it is now in their interest to comply with anything else governments ask them to do.
  - anshumankmr 5y ago
    How do you think will the lawyers be able to prove Apple is a monopoly when Android exists?
    - diebeforei485 5y ago
      The mere existence of a competitor doesn't matter in competition law. Other oil companies existed, but the government still broke up Standard Oil. Other browsers existed, but the government still made Microsoft make certain API's available to other browser makers.
      - anshumankmr 5y ago
        This is a false equivalence. Standard Oil was at the top. Apple is clearly not. In the USA, they may very well be, but I doubt the margin is very high. https://images.app.goo.gl/hLr1zAGs7Wq5TG46A https://images.app.goo.gl/hLr1zAGs7Wq5TG46A A lawyer will say that Apple though in the lead as a brand, is still second to Android in terms of market share, no matter how fragmented the market is. This might be the same in other western countries as well. But overall Android would still have more devices running it than iOS.

swiley 5y ago

Nothing except Apple saying you could trust them. People were stupid enough to accept that and now even the trust is gone.

fshbbdssbbgdd 5y ago

I think “we don’t have the machinery to do that” is an effective argument in the real world when someone asks you do to something. I’m not sure if it matters legally (lawyers sometimes use vague phrases like “reasonable effort”), but it definitely affects how strongly people will pressure you to do things, and how likely you are to acquiesce to that pressure. The scope of the change Apple would need to make to scan your photos arbitrarily just got a lot smaller. The number of engineers who would need to be “in the know” to implement this change got smaller. The belief from governments that Apple has the option of doing this got stronger. The belief among Apple’s own management team that they can do this got stronger.

snowwrestler 5y ago
This is very well put.

croutonwagon 5y ago

In years previous, take the San Bernadino shooter for instance, Apple argued in the court of law that creating backdoors or reversible encryption was insecure and also subject to exploits by malicious actors, and thus not reasonable and was "unreasonably burdensome". As well, they made the argument that compelling them to do write back doors also violated the first amendment. It was most likely a winning strategy that the FBI actively avoided getting rulings on and found a workaround. What apple is creating here is an avenue for the FBI/NSA/Alphabet agency to create a FISA warrant and NDL to mandate hits on anything. The argument its gotta be pre-icloud upload or subject to manual review or on some arbitrary threshold is something is just the marketing to get the public to accept it. All of that can easily be ordered to be bypassed. So it can be a scan, single hit for x, report. Ill take the downvotes, but if anything, someone more conspiracy minded could easily take this as a warrant canary. Given the backlash apple ahs faced and ignored, it doesnt make much good business sense for them not to back off unless they are A) betting on it being a vocal minority to resorts to action (which is entirely possible, especially given the alternatives and technical hurdles to get to a suitable alternative) B) Being pressured by governments now. (also entirely possible given their history with the FBI and previous investigations). [1] https://www.rpc.senate.gov/policy-papers/apple-and-the-san-bernardino-terror-attack https://www.rpc.senate.gov/policy-papers/apple-and-the-san-b... [2] https://en.wikipedia.org/wiki/FBI%E2%80%93Apple_encryption_dispute#Apple's_opposition_to_the_order https://en.wikipedia.org/wiki/FBI%E2%80%93Apple_encryption_d...

nicce 5y ago
> What apple is creating here is an avenue for the FBI/NSA/Alphabet agency to create a FISA warrant and NDL to mandate hits on anything. The argument its gotta be pre-icloud upload or subject to manual review or on some arbitrary threshold is something is just the marketing to get the public to accept it. Why would they make things even more complicated with limited access, since they can already access everything in cloud? Let’s leave out the argument for expanding scan to whole device. If that is what happens, then people start really discarding their phones.
- klipklop 5y ago

int_19h 5y ago

The politics of it is very different, and that's where the danger lies: https://news.ycombinator.com/item?id=28239506 https://news.ycombinator.com/item?id=28239506 I think that quite a few engineers are too focused on the technical aspects of it, and specifically on all those "barriers to misuse" that Apple claims to have in place. But it'll be much easier to remove the barriers once the system as a whole is in place.

judge2020 5y ago
The reason we're focused on the state of it now is that we can switch at any time - especially if those barriers are shown to be ineffective or are removed at some point.
- int_19h 5y ago
  The real threat here is legal, not technical. Think mandatory on-device scanning as condition of access to the hardware market.
  - judge2020 5y ago
    That just ties back into "be afraid of what it could become", and isn't dependent on Apple making this system - Congress could have forced PhotoDNA to be shipped with phones since the inception of PhotoDNA in 2011[0]. 0: https://doi.org/10.1016/S0262-4079(11)60791-4 https://doi.org/10.1016/S0262-4079(11)60791-4
    - int_19h 5y ago
      Of course it's not dependent on it being Apple. It could just as well have been Google. And the difference is that Apple gave them what they needed to mandate this while still claiming that they preserve privacy.

sillystuff 5y ago

> What I don't get is what prevented these things from happening last month? Apple controls the hardware, the software, and the cloud services... Yes, proprietary black-box hardware and software is poor from a user privacy perspective. But, If Apple began on-device scanning of content, I'd imagine eventually someone would notice the suspicious activity and investigate. With Apple's announcement, the scanning will just be something that Apple devices do. Nothing to worry about. And, no way for anyone to independently verify that the scope of the content being scanned has not been secretly increased. As for icloud, if your content is not encrypted on the device in a manner where only you have the keys, any cloud storage is suspect for scanning / data mining. But, on-device scanning is a back door for e2e encryption-- even on device encryption with keys only you control is thwarted.

shuckles 5y ago
How do you think Apple will increase the scope of what’s scanned without every person with Ghidra skills not noticing?
- sillystuff 5y ago
  If the exchange with Apple is encrypted / interleaved with other traffic to icloud, how would you know that there isn't new classes of scanning being done? I'll be very surprised if similar tech is not lobbied for as a backstop to catch DRM-free media files played on devices we "own". And, it seems far more probable than not that police will demand this capability be used to help address more crimes. The problem here is that crimes can mean speaking out against an oppressive regime. Being targeted for having the wrong political views (think McCarthyism in the United States or the US backed murder of a million people in Indonesia for affiliating with the "wrong" political party). Etc. History is awash with political abuse of "out groups" perpetrated by tin pot dictators all the way to presidents and PMs of major world powers. And, it sets the precedent that e2e encryption is not an excuse for a provider to not provide private customer data to the authorities-- a back-door can be installed, "Just do what Apple did."
JimDabell 5y ago
> no way for anyone to independently verify that the scope of the content being scanned has not been secretly increased. This seems like the easiest thing out of the lot to verify. The way that this system is designed to work is that when uploading to iCloud Photos, images have a safety voucher attached to them. If Apple secretly expanded this to scan more than just iCloud Photos, they would have to either a) upload all the extra photos, b) add a new mechanism to upload just the vouchers, or c) upload “fake” photos to iCloud Photos with the extra vouchers attached. None of these seem particularly easy to disguise. Your concern is completely understandable if you are starting from the premise that Apple are scanning photos then uploading matches. I think that’s how a lot of people are assuming this works, but that’s not correct. Apple designed the system in a very different way that is integrated into the iCloud upload process, and that design makes it difficult to expand the scope beyond iCloud Photos surreptitiously. Could Apple build a system to secretly exfiltrate information from your phone? Of course. They could have done so since the first iPhone was released in 2007. But this design that they are actually using is an awful design if that’s what they wanted to do. All of their efforts on this seem to be pointed in the exact opposite direction.

NicoJuicy 5y ago

> They could have already scanned our files because they already have full control over the entire ecosystem They did do it in emails since 2019: https://www.indiatoday.in/technology/news/story/apple-has-been-scanning-icloud-mail-for-child-exploitation-images-since-2019-1844732-2021-08-24 https://www.indiatoday.in/technology/news/story/apple-has-be...

noptd 5y ago

This entire argument is a non sequitur and comes up like clockwork every time this issue is discussed. It's the metaphorical equivalent of saying "well someone could've snuck in through the open window. Let's just assume they did and leave the doors open as well". How about instead we push back against Apple further shifting the Overton window on how acceptable it is for companies to run intrusive services on hardware we own?

shuckles 5y ago
It’s not a non sequitur. The comment is engaging with a series of rhetorical questions that imagine a slippery slope by observing that very little has changed about the trust model between iPhone users and their devices. If you are convinced Apple is slipping, then it is worthwhile to be able to answer how their position today is different than it was last month. That is of course a different question than whether their position last month was acceptable, and maybe people are realizing it was not. As a concrete example, if you think the proposal introduces new technical risks, then if Apple announces they made a mistake and will instead scan entirely on the server, you may be satisfied. However, I’d argue that since no new technical risk has been introduced, your conclusions should not change. I’d argue that the incorrect characterization of Apple’s announcement as scanning all the files on your phone with no control has shifted the Overton window more than what was actually proposed. Politicians who are none the wiser probably believe that’s what Apple actually built, even though it’s not.
- noptd 5y ago
  I disagree - it's a distraction from the larger issue at hand. > I’d argue that the incorrect characterization of Apple’s announcement as scanning all the files on your phone with no control That's a strawman - few if any are arguing that the system will read all of your files out of the gate. >since no new technical risk has been introduced, This assumption doesn't reflect reality. Introducing a brand new system built specifically for client-side scanning absolutely adds technical risks, if nothing else then by the sheer fact that it's adding another attack vector on your phone. Not to mention the fact that all it would take is a change in policy and a few trivial updates (a new event trigger, directory configs, etc) for this system to indeed scan any file on your device.

rkagerer 5y ago

What I don't get is what prevented these things from happening last month? Apple controls the hardware, the software, and the cloud services... Simple: Money. Their response to any such demands would be (and has been) "we don't have the capability to do what you're asking". No judge is going to burden them to spend their own dime to build a massive new feature like this and deploy it to every phone out there to comply with a demand arising from a prosecutor of an individual case. No government is going to pony up the money to reimburse them to do it (not even getting into the PR optics). That leaves it happening only if 1) they decide to do it themselves, or 2) government(s) legislate they must. So far #2 hasn't happened. Politicians had no basis of reference to point to and say "Your competitor(s)' doing that, you should too". But now that #1 occurred, it will normalize this nonsense and pave the way for #2.

WillPostForFood 5y ago
Their response to any such demands would be (and has been) "we don't have the capability to do what you're asking". No judge is going to burden them to spend their own dime to build a massive new feature like this and deploy it to every phone out there to comply with a demand arising from a prosecutor of an individual case. Government does not care one bit about how much it costs or if it is even possible. They demand the data with an ultimatum: deliver it as we requested by our deadline or we send in our IT people to take it. Sorry (not) if it takes your whole company down while we plugin in our own servers in your datacenter to take your data.
- rkagerer 5y ago
  Doesn't work if the data of interest is not there for the taking. And a judge will not compel beyond what they consider reasonable. Having the feature already in place dramatically shifts the bar.
shuckles 5y ago
Their response to such demands has not been we are technically incapable of doing what’s requested. The demand from the FBI in the San Bernardino case was a very small change to passcode retry constants, because the terrorist’s device did not have a Secure Element.

judge2020 5y ago

> They could have already scanned our files because they already have full control over the entire ecosystem. Apple barely submits any CSAM[0]: > According to NCMEC, I submitted 608 reports to NCMEC in 2019, and 523 reports in 2020. In those same years, Apple submitted 205 and 265 reports (respectively). It isn't that Apple doesn't receive more picture than my service, or that they don't have more CP than I receive. Rather, it's that they don't seem to notice and therefore, don't report. 0: https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On...

xunn0026 5y ago

The existing law and user agreement also sets your rights. Apple might have done what you say last month, illegally. Now it's in the user agreement and they can do it legally, at scale. This literally creates a precedent.