12 ms·

This technology will soon be out of Apple’s control. Higgins correctly highlights the immense pressure Apple will get from governments and other actors to bend

This technology will soon be out of Apple’s control. Higgins correctly highlights the immense pressure Apple will get from governments and other actors to bend the technology and use it for something else than csam. It will happen, people are probably already thinking how to apply such pressure. Sooner or later Apple will cave in and they will have only themselves to blame when freedom supports in Sudan or LGBTQ activists in Saudi Arabia will be jailed. The issue here is not where to draw the line, but who will draw it (spoiler: not Apple).

35803288 5y ago
Ten years from now we will read a leaked document stating that US authorities requested FISA Courts warrants (or similar) and made Apple scan for things other than csam. It already happened. Court orders are easier than hacks once you iron out the legal opinions.
- sparker72678 5y ago
  Even if that's true, this method at least drops some part of the process on the client where it can be inspected, vs. all scanning happening in the cloud, entirely behind closed doors. Does that mean nothing bad can happen? No. But it does mean that when something changes, we at least know something changed.
  - jtbayly 5y ago
    The database will change every update. It’s not like the NCEMC db already has all possible CSAM. Every time they find more, they add it to the DB. Thus, it will probably change every time Apple pushes an OS update.
- Lamad123 5y ago
  10 years seems a little too much for US gov's patience
cyanite 5y ago
Since it’s speculation at this point, I’d say we deal with it if and when it happens. I also don’t see how this doesn’t apply even worse to doing cloud side scanning, like companies otherwise do. Is that out of control? Is there any evidence of that?
- JohnFen 5y ago
  It's worse because the cloud is someone else's computer. When you're in someone else's house, you go by their rules. Your devices, however, are your own house.
  - eternalban 5y ago
    > the cloud is someone else's computer. So by your logic, if I use a post office box, that means the postal service has the right to open all my packages?
    - Jtsummers 5y ago
      The USPS is restricted by the Constitution and other legislation on what it can do. From: https://www.uspis.gov/wp-content/uploads/2019/05/USPIS-FAQs.pdf https://www.uspis.gov/wp-content/uploads/2019/05/USPIS-FAQs.... > 4. Can Postal Inspectors open mail if they feel it may contain something illegal? First-Class letters and parcels are protected against search and seizure under the Fourth Amendment to the Constitution, and, as such, cannot be opened without a search warrant. If there is probable cause to believe the contents of a First-Class letter or parcel violate federal law, Postal Inspectors can obtain a search warrant to open the mail piece. Other classes of mail do not contain private correspondence and therefore may be opened without a warrant. Companies hosting your data aren't similarly restricted (though if the US government wants access then they'd be restricted by the Constitution and legislation again). A company's ability to look at whatever you give them is only restricted by your contract with them and the technical limitations created by how you share it (upload encrypted files where they don't have the key? they can't really do much). They may have some legal restrictions on some kinds of data, but it's not going to be uniform across the globe so you'll have to take care with which companies you choose to host your unencrypted data.
ohazi 5y ago
Why does anyone even assume that a bad actor would need to apply pressure? These databases are unauditable by design -- all they'd need to do is hand Apple their own database of "CSAM fingerprints collected by our own local law enforcement that are more relevant in this region" (filled with political images of course), and ask Apple to apply their standard CSAM reporting rules. That's it... Tyranny complete.
- judge2020 5y ago
  https://www.neowin.net/news/apple-says-its-new-child-safety-feature-will-look-for-images-flagged-in-multiple-countries/ https://www.neowin.net/news/apple-says-its-new-child-safety-... Yes, they could change it at any time. But this was always the case with all software that implements OTA updates.
- sparker72678 5y ago
  1) The DB must be updated on both the server and the client. Apple's solution requires the DBs to match, essentially. So you can't update the DB without everyone knowing it was changed. 2) Apple has trillions of dollars to lose by selling out to a shitty change like that. Do those things mean nothing bad can come of it? Hell no. But, right now we have FISA courts and silent warrants sucking in data without anyone knowing or being able to talk about it. It's not like we're a panacea at the moment. Apple's approach creates a possibility of slowing down the politics already trying to move against E2EE data. This is a political fight, and if we all act like ideologues we're going to lose it all in the end.
  - xkcd-sucks 5y ago
    > Apple's approach creates a possibility of slowing down the politics already trying to move against E2EE data. This is "appeasement" or "Danegeld", and we all know how that works out in the end
  - ScoobleDoodle 5y ago
    Apples move gives away another piece of the puzzle to mass monitoring and surveillance. Apples move brings it one step closer for FISA courts and silent warrants to have access beyond what we send over the network to now what resides on our phones. Apple is giving mass surveillance a foot hold into living on and monitoring data on our personal phones. Apples has created the back door for increased surveillance: https://www.eff.org/deeplinks/2021/08/if-you-build-it-they-will-come-apple-has-opened-backdoor-increased-surveillance https://www.eff.org/deeplinks/2021/08/if-you-build-it-they-w... Tell Apple don't scan our phones: https://act.eff.org/action/tell-apple-don-t-scan-our-phones https://act.eff.org/action/tell-apple-don-t-scan-our-phones
naravara 5y ago
> Higgins correctly highlights the immense pressure Apple will get from governments and other actors to bend the technology and use it for something else than csam. It will happen, people are probably already thinking how to apply such pressure. Sooner or later Apple will cave in and they will have only themselves to blame when freedom supports in Sudan or LGBTQ activists in Saudi Arabia will be jailed. I'm having trouble seeing how Apple's actions make this any more or less likely. It's not like matching photos is some esoteric concept that no repressive government has ever thought of before. It's not even like it's particularly hard. Apple's implementation is the most privacy sensitive way of doing it, but if the rules were going to come down they were going to come down, and they'd be implemented in less privacy sensitive ways.
- robertoandred 5y ago
  If Sudan or Saudi Arabia wants to arrest people, there are much easier ways. What “freedom” or “LGBTQ” photos would you even search for?
- simondotau 5y ago
  Apple can also choose to exit that country. There aren't many countries which Apple would even consider risking its global reputation in order to retain that market. US, China, Europe, maybe the UK. That's about it. If Saudi Arabia or Sudan tried to turn the screws, the business case for Apple is absolutely clear-cut: they leave. This isn't even up for debate. There's far too much at risk globally than there is to gain domestically from compliance. Not only do they avoid serious damage to their global reputation (something they'll be extremely sensitive to, as the last two weeks have taught them) it would represent a massive opportunity for Apple to earn weeks of free media coverage that aligns with their security narrative.
ScoobleDoodle 5y ago
Tell Apple don't scan our phones: https://act.eff.org/action/tell-apple-don-t-scan-our-phones https://act.eff.org/action/tell-apple-don-t-scan-our-phones
greyface- 5y ago
Exactly. Governments tend to accept "we lack the technical capability to comply with your request" (unless said capability is legally mandated, e.g. so-called "lawful intercept"). They do not tend to accept "we possess the technical capability to comply with your request but choose not to do so".
- deleted 5y ago
  [deleted]
simondotau 5y ago
I realise that you're talking about global Governments, but when it comes to the United States, Government pressure cannot compel Apple to expand the on-device searching because that would be an unequivocal violation of the 4th Amendment of the US Constitution. Because it is a search of your private property compelled by the Government. (After a photo is uploaded to a cloud service, a search of photos stored on servers doesn't enjoy the same 4A protection as this falls under the so-called "third party doctrine".) (Apple searching for CSAM is also not a 4A violation because it was Apple's free choice as a private company to do so, and you will have agreed to it as part of the Terms of Service of the next version of iOS.)
LdSGSgvupDV 5y ago
It seems like big techs are taking the place of court in some domains. They draw the lines and execute the rules. Even more, it is almost impossible to know where are the real lower and upper bound of rules. There are already some examples (e.g. app is banned for unknown/vague reasons).
raxxorrax 5y ago
Western governments had enough room to select a different way. Instead we are captured by a destructive war on terror surveillance ambitions. Do you think those defense and security contractors will ever go away? That they won't summon dangers if they have nothing else to do? Here governments could have opted for a contrast but they neglected these opportunities.

35803288 5y ago

Ten years from now we will read a leaked document stating that US authorities requested FISA Courts warrants (or similar) and made Apple scan for things other than csam. It already happened. Court orders are easier than hacks once you iron out the legal opinions.

sparker72678 5y ago
Even if that's true, this method at least drops some part of the process on the client where it can be inspected, vs. all scanning happening in the cloud, entirely behind closed doors. Does that mean nothing bad can happen? No. But it does mean that when something changes, we at least know something changed.
- jtbayly 5y ago
  The database will change every update. It’s not like the NCEMC db already has all possible CSAM. Every time they find more, they add it to the DB. Thus, it will probably change every time Apple pushes an OS update.
Lamad123 5y ago
10 years seems a little too much for US gov's patience

cyanite 5y ago

Since it’s speculation at this point, I’d say we deal with it if and when it happens. I also don’t see how this doesn’t apply even worse to doing cloud side scanning, like companies otherwise do. Is that out of control? Is there any evidence of that?

JohnFen 5y ago
It's worse because the cloud is someone else's computer. When you're in someone else's house, you go by their rules. Your devices, however, are your own house.
- eternalban 5y ago
  > the cloud is someone else's computer. So by your logic, if I use a post office box, that means the postal service has the right to open all my packages?
  - Jtsummers 5y ago
    The USPS is restricted by the Constitution and other legislation on what it can do. From: https://www.uspis.gov/wp-content/uploads/2019/05/USPIS-FAQs.pdf https://www.uspis.gov/wp-content/uploads/2019/05/USPIS-FAQs.... > 4. Can Postal Inspectors open mail if they feel it may contain something illegal? First-Class letters and parcels are protected against search and seizure under the Fourth Amendment to the Constitution, and, as such, cannot be opened without a search warrant. If there is probable cause to believe the contents of a First-Class letter or parcel violate federal law, Postal Inspectors can obtain a search warrant to open the mail piece. Other classes of mail do not contain private correspondence and therefore may be opened without a warrant. Companies hosting your data aren't similarly restricted (though if the US government wants access then they'd be restricted by the Constitution and legislation again). A company's ability to look at whatever you give them is only restricted by your contract with them and the technical limitations created by how you share it (upload encrypted files where they don't have the key? they can't really do much). They may have some legal restrictions on some kinds of data, but it's not going to be uniform across the globe so you'll have to take care with which companies you choose to host your unencrypted data.

ohazi 5y ago

Why does anyone even assume that a bad actor would need to apply pressure? These databases are unauditable by design -- all they'd need to do is hand Apple their own database of "CSAM fingerprints collected by our own local law enforcement that are more relevant in this region" (filled with political images of course), and ask Apple to apply their standard CSAM reporting rules. That's it... Tyranny complete.

judge2020 5y ago
https://www.neowin.net/news/apple-says-its-new-child-safety-feature-will-look-for-images-flagged-in-multiple-countries/ https://www.neowin.net/news/apple-says-its-new-child-safety-... Yes, they could change it at any time. But this was always the case with all software that implements OTA updates.
sparker72678 5y ago
1) The DB must be updated on both the server and the client. Apple's solution requires the DBs to match, essentially. So you can't update the DB without everyone knowing it was changed. 2) Apple has trillions of dollars to lose by selling out to a shitty change like that. Do those things mean nothing bad can come of it? Hell no. But, right now we have FISA courts and silent warrants sucking in data without anyone knowing or being able to talk about it. It's not like we're a panacea at the moment. Apple's approach creates a possibility of slowing down the politics already trying to move against E2EE data. This is a political fight, and if we all act like ideologues we're going to lose it all in the end.
- xkcd-sucks 5y ago
  > Apple's approach creates a possibility of slowing down the politics already trying to move against E2EE data. This is "appeasement" or "Danegeld", and we all know how that works out in the end
- ScoobleDoodle 5y ago
  Apples move gives away another piece of the puzzle to mass monitoring and surveillance. Apples move brings it one step closer for FISA courts and silent warrants to have access beyond what we send over the network to now what resides on our phones. Apple is giving mass surveillance a foot hold into living on and monitoring data on our personal phones. Apples has created the back door for increased surveillance: https://www.eff.org/deeplinks/2021/08/if-you-build-it-they-will-come-apple-has-opened-backdoor-increased-surveillance https://www.eff.org/deeplinks/2021/08/if-you-build-it-they-w... Tell Apple don't scan our phones: https://act.eff.org/action/tell-apple-don-t-scan-our-phones https://act.eff.org/action/tell-apple-don-t-scan-our-phones

naravara 5y ago

> Higgins correctly highlights the immense pressure Apple will get from governments and other actors to bend the technology and use it for something else than csam. It will happen, people are probably already thinking how to apply such pressure. Sooner or later Apple will cave in and they will have only themselves to blame when freedom supports in Sudan or LGBTQ activists in Saudi Arabia will be jailed. I'm having trouble seeing how Apple's actions make this any more or less likely. It's not like matching photos is some esoteric concept that no repressive government has ever thought of before. It's not even like it's particularly hard. Apple's implementation is the most privacy sensitive way of doing it, but if the rules were going to come down they were going to come down, and they'd be implemented in less privacy sensitive ways.

robertoandred 5y ago
If Sudan or Saudi Arabia wants to arrest people, there are much easier ways. What “freedom” or “LGBTQ” photos would you even search for?
simondotau 5y ago
Apple can also choose to exit that country. There aren't many countries which Apple would even consider risking its global reputation in order to retain that market. US, China, Europe, maybe the UK. That's about it. If Saudi Arabia or Sudan tried to turn the screws, the business case for Apple is absolutely clear-cut: they leave. This isn't even up for debate. There's far too much at risk globally than there is to gain domestically from compliance. Not only do they avoid serious damage to their global reputation (something they'll be extremely sensitive to, as the last two weeks have taught them) it would represent a massive opportunity for Apple to earn weeks of free media coverage that aligns with their security narrative.

ScoobleDoodle 5y ago

Tell Apple don't scan our phones: https://act.eff.org/action/tell-apple-don-t-scan-our-phones https://act.eff.org/action/tell-apple-don-t-scan-our-phones

greyface- 5y ago

Exactly. Governments tend to accept "we lack the technical capability to comply with your request" (unless said capability is legally mandated, e.g. so-called "lawful intercept"). They do not tend to accept "we possess the technical capability to comply with your request but choose not to do so".

deleted 5y ago
[deleted]

simondotau 5y ago

I realise that you're talking about global Governments, but when it comes to the United States, Government pressure cannot compel Apple to expand the on-device searching because that would be an unequivocal violation of the 4th Amendment of the US Constitution. Because it is a search of your private property compelled by the Government. (After a photo is uploaded to a cloud service, a search of photos stored on servers doesn't enjoy the same 4A protection as this falls under the so-called "third party doctrine".) (Apple searching for CSAM is also not a 4A violation because it was Apple's free choice as a private company to do so, and you will have agreed to it as part of the Terms of Service of the next version of iOS.)

LdSGSgvupDV 5y ago

It seems like big techs are taking the place of court in some domains. They draw the lines and execute the rules. Even more, it is almost impossible to know where are the real lower and upper bound of rules. There are already some examples (e.g. app is banned for unknown/vague reasons).

raxxorrax 5y ago

Western governments had enough room to select a different way. Instead we are captured by a destructive war on terror surveillance ambitions. Do you think those defense and security contractors will ever go away? That they won't summon dangers if they have nothing else to do? Here governments could have opted for a contrast but they neglected these opportunities.