10 ms·

The clear distinction being that iCloud Mail scanning doesn't happen on device. For my part, all Apple needs to do is move CSAM scanning to the cloud. No servi

by YPPH 5y ago

The clear distinction being that iCloud Mail scanning doesn't happen on device. For my part, all Apple needs to do is move CSAM scanning to the cloud. No service provider can be expected to keep images of child abuse on their servers. Apple would join myriad cloud service providers in scanning for and reporting such material. My problem is the use of my own device to run the scan. It's a waste of system resources. Presumably, a trivial software update down the line could expand its ambit to locally stored files. And we have the issue of Apple being compelled to run searches for non-CSAM hashes.

jnwatson 5y ago
The point was so they wouldn't need to see the plaintext of the photos on their servers. Somebody needs to invent a FHE (fully homomorphic encryption) CSAM algorithm, so that Apple could scan encrypted photos for badness on the cloud.
- rfd4sgmk8u 5y ago
  Not Cool, now we can enforce all kinds of arbitrary censorship on encrypted data. Be careful what you wish for.
  - dhosek 5y ago
    Also, the Apple system allows for a way of verifying matches to screen for false positives which the encrypted approach would not.
- YPPH 5y ago
  Are photos uploaded to iCloud encrypted client-side? I suggest the answer is in the negative, given the ability to look at them on the web, and the ability to reset your iCloud password using 2FA. These suggest Apple has a copy of the decryption keys, which I'm happy for them to use to scan files I have stored on their servers, for CSAM, or, within reason, anything.
  - amznthrwaway 5y ago
    Yes, they’re encrypted client-side, then unencrypted in browser. This is relatively easy to verify if you do not trust an anonymous comment, and I suggest you do so.
  - rovr138 5y ago
    > Are photos uploaded to iCloud encrypted client-side? Well, I can share a link to them and others can see them, including anonymous users. Just leads me to believe that either they aren’t or if they are, they have the key.
  - mulmen 5y ago
    This boils down to “I have nothing to hide.” I encourage you to review https://www.schneier.com/essays/archives/2006/05/the_eternal_value_of.html https://www.schneier.com/essays/archives/2006/05/the_eternal... and consider if you really feel this way.
- howaboutnope 5y ago
  Alternatively, someone could realize that existing CSAM that's so widespread it's in the database basically equals water under the bridge, whereas creating CSAM that isn't in the DB requires to harm more children. Poverty and power imbalance is what harms children the most, by far. But we can't tackle that without stepping on the toes of greed, so we do circus instead.
  - JKCalhoun 5y ago
    You say water under the bridge, others will say where there's smoke there's fire.
    - howaboutnope 5y ago
      But this isn't smoke, this is a photo of smoke. And as I said, the raging fires all of us can see in plain view go mostly unchecked. Where there is a photo of smoke, there may be fire, but where there is obviously a huge blazing fire, there surely is fire.
- oconnore 5y ago
  In some ways, I think privacy advocates could be shooting themselves in the foot here. I think that mainstream storage encryption (mainstream as in WhatsApp and Signal, as opposed to encrypting your NAS using linux commands) is very important for privacy, and that deploying it without running into CSAM objections and legislation is a Sisyphean task for for-profit businesses. Messaging seems to be more legally defensible than storage from a business perspective. If you take as an axiom that mainstream businesses will be forced to protect themselves against contributing to CSAM distribution, then the choice to offer encrypted cloud storage to non-technical end users REQUIRES doing the scan on a trusted computer (Apple has chosen the phone itself). I think the arguments that this can be abused are very real, but it's worth talking about how to fix that, because I think the alternative might be sacrificing E2EE cloud storage in the mainstream (as has happened with every other mainstream company). Perhaps more thought should be put into making this process auditable by the device owner (or by a trusted 3rd party -- say the EFF). Or perhaps the scanning could be federated -- say I don't want Apple doing that, but I might trust a privacy oriented non-profit to "certify" to Apple that my personal photo album is CSAM-free. Can that 3rd party scan be blinded, such that I send data that is representative of my images, but I've already anonymized my photos using a transformation? Could we audit (similar to certificate transparency): 1) What data from the device is being scanned? What data is being uploaded? 2) What "hashes" are being matched against, and how are those changing over time? Can the data lineage of the NCMEC database be audited? Would that pick up malicious hashes injected into the database? Generally, I think our privacy paradigm needs to be built in such a way that it can actually be deployed in our policy environment. More realpolitik, less ethical grandstanding.
- anonuser123456 5y ago
  FHE doesn't let the other party access the results of algorithms run on your data. Other parties can run an algorithm on your encrypted data, but only you can interpret the results.
  - jnwatson 5y ago
    That's a good point. The device with the key would still have to forward the results.
rdudek 5y ago
This is correct. I have no problems of them scanning my icloud mail or photos. I just don't like the idea of there being a system level scanner for a hash database on my device assuming that I'm guilty until proven innocent. I was going to upgrade my iPhone this fall to latest model, now I'm very hesitant to do so and looking at alternatives.
- istingray 5y ago
  You might find this helpful in your search -- HN Discussion of open source phones: https://news.ycombinator.com/item?id=28164208 https://news.ycombinator.com/item?id=28164208
- robertoandred 5y ago
  But you do like the idea of a server-level scanner for a hash database assuming that you're guilty until proven innocent?
  - smoldesu 5y ago
    Nobody does, but that's the current status-quo anyways. Almost every cloud-based service provider will do this anyways, Apple's only change is that they've moved their surveillance from their computers to mine. If I can't know how secure my phone is at all times, then I frankly can't feel comfortable using it.
  - JKCalhoun 5y ago
    Yeah, I don't get the cloud/device distinction people are making. Increasingly they are one and the same (if you know what I mean).
    - ramses0 5y ago
      ODINN => On Device, Intrusive, Neural Network. You cannot photocopy a dollar bill. Kinda makes sense. https://en.wikipedia.org/wiki/EURion_constellation https://en.wikipedia.org/wiki/EURion_constellation Now imagine a "live" neural network running on your phone or camera that prevents you from taking pictures of the Eiffel Tower at night due to "Suspected Copyright Abuse". https://www.snopes.com/fact-check/photographs-of-eiffel-tower-at-night/ https://www.snopes.com/fact-check/photographs-of-eiffel-towe... Next imagine that a local government issues a digital "All Points Bulletin" which scans all pictures on all phones for a "person of interest in a crime (against the state)", conveniently associated with the emergency alert broadcast network. First they came: https://en.wikipedia.org/wiki/First_they_came_ https://en.wikipedia.org/wiki/First_they_came_...
joshstrange 5y ago
Genuine question. I know this is not the situation we find ourselves in but, if your options were on-device scanning for CSAM+encrypted iCloud Photos (that Apple could not decode) OR cloud-based scanning of iCloud Photos which would you pick? Are you still opposed to all on-device scanning? Edit: Thank you all for your replies. I don't share the exact same concerns as some of you but I appreciate you sharing your thoughts/reasoning. I really did what to know what people thought about this in the context of an E2E iCloud-* and it's been hard to pick that out of all the other discussions.
- YPPH 5y ago
  I'd pick the latter. I doubt iCloud will ever be a zero-knowledge service.
- smoldesu 5y ago
  Probably the latter, because frankly E2E encryption means very little to me when one end is the world's most powerful company.
- haaserd 5y ago
  I'd happily pick cloud based scanning. At least that way I can opt out by not using their cloud. I have serious doubts that choosing not to use iCloud will actually stop Apple from running scanning code on my device. Having a capability pretty much ensures that it will be used, and likely misused.
- echelon 5y ago
  My house shouldn't be full of eyes and ears. CSAM today, political materials another. This could also be used to identify whistle blowers, reporters' sources, and more. I'd hate to have gay porn on my iPhone in a Sharia law state. Or images of tank man in mainland China. Imagine when the detector extends to not just files. Things typed or said. This is a steep cliff, and we're drawing closer to the edge.
  - zepto 5y ago
    > Or images of tank man in mainland China. The Chinese government forcibly installs spyware on people’s phones today. What Apple does or doesn’t do is of no consequence to them.
deleted 5y ago
[deleted]
throwawaymanbot 5y ago
The surveillance infrastructure IS the point. NO point just doing it in the cloud. It HAS to be on device.
c7DJTLrn 5y ago
You've hit the nail on the head for me. The part that offends me is that the scanning is happening on my device that I paid for, and had no knowledge of the eventual introduction of when I bought the phone. If they want to scan cloud storage, by all means. But I shouldn't have to have my device bogged down with this extra pointless computation.
- zepto 5y ago
  The idea that it’s going to slow your device down is an odd one.
Infernal 5y ago
> Presumably, a trivial software update down the line could expand its ambit to locally stored files. And backing up one level, this is why I’m finally working hard to take control of my devices and data from companies like Apple. This most recent episode shows that “a trivial software update” such as the one in iOS 14.3 can introduce this on-device scanning where non existed before. We knew it was possible of course, but now we have a clear cut example of it. They’ve lost my trust and that’s that.
- MrWiffles 5y ago
  > [...] They’ve lost my trust [...] Exactly how I feel. I used to cautiously kinda trust Apple's so-called "commitment" to privacy, but this change just shows us that any such promises are purely theater. Now that the damage is done, I doubt it can be undone, especially in my case. My opinion on Apple's credibility has crashed through the floor. In my opinion, they're now as untrustworthy as Google or Facebook.
  - istingray 5y ago
    For me it's a step further. I cannot trust any company for more than privacy theater. Apple was simply the last to fall. I'm starting to look at companies to see if they meet these must-haves: 1. E2E encryption 2. Open source hardware and software Everything else is "nice to have". The current options out there are missing a lot of "nice to haves", but they will get better. I'll be a vocal customer telling them what I want.
- mbjdesign 5y ago
  “Trust” is entirely my issue with Apple too. I spent some time - and even wrote a post on my site - trying to clarify my thoughts and the steady erosion of trust seemed to be what I most object to. I too am trying to take control and responsibility for the computers I use now. Proving tricky, but baby steps…
  - istingray 5y ago
    Care to share the link to the post on your site?
zepto 5y ago
> Presumably, a trivial software update down the line could expand its ambit to locally stored files. Sure but this is equally true now as it will be after they deploy the CSAM detector. It’s not a real argument against the detector.
skoskie 5y ago
Doing so would sacrifice privacy. I'll take the hit on my battery.
_trampeltier 5y ago
I asked in another thread (and got downvoted), if Apple does it on the user devices to save money. How much would it cost to scan everything on the serverside?
mulmen 5y ago
Ideally they wouldn’t be able to scan my iCloud photos at all. There’s real value in being able to maintain privacy while also gaining the benefits of live, remote storage and backup. The generous interpretation is that Apple moved this on-device to preserve user privacy. They just miscalculated the response to an admitted backdoor.
nitrogen 5y ago
My problem is the use of my own device to run the scan. It's a waste of system resources. I'll restate what I posted on another thread about this [0]. There should be a clear, bright line here. ---- In all these threads everyone is coming close to the crux of the issue, but I want to restate it in clearer terms: There is a sacrosanct line between "public" and "private," "mine" and "yours." That line cannot be crossed by Western governments without a warrant. Cloud computing has deliberately blurred this line over time. This on-device scanning implementation blows right past that line. Our tools before the computing revolution, and our devices after, become a part of us. Our proprioception extends to include them as part of "self." A personal device -- a tool that should be wholly owned and wholly dependable, like pen and paper -- that betrays its user, is a self that betrays itself. [0]: https://news.ycombinator.com/item?id=28162412 https://news.ycombinator.com/item?id=28162412
- nostromo 5y ago
  You're exactly right. We have a word for what Apple is installing on your device: spyware. And it's worse than typical spyware in that it's using your device to spy on you and report its findings back to law enforcement.
- Terretta 5y ago
  Framing as “on device scanning” is contrary to facts. It’s the client SDK operating the same algo on an upload event image at the client end of the upload API as it would run at the server end, just not having to decrypt it to do it. But to your theme — there’s no clear bright line in client SDKs. Devs do not think of these or handle these as if they belong to the user. If you study the cesspool of user tracking and other privacy violations on iOS devices, it’s mostly developers using the same client-side SDKs across their apps, and users none the wiser. Sometimes devs also have no idea. At least, they profess surprise and drop these SDKs when the user behavior harvesting is pointed out to them. For whatever reason, everyone’s OK with this. Perhaps the line is too blurred between client and server, thanks to emergence of concepts like node.js (“run same code on client or server”) that make this sound like a feature. Or maybe it really shouldn’t matter where the code runs provided it respects boundaries, or better, those boundaries of intentionality and access are technically guaranteed inviolate.
wayneftw 5y ago
This is exactly what I complained about in past threads and nearly nobody agreed with me there. Seems like the tide has changed! This is my only complaint. I don't care what you do on your own cloud, don't try and force my device to do the heavy lifting especially when the activity is not in my own interest.

by YPPH 5y ago

jnwatson 5y ago

The point was so they wouldn't need to see the plaintext of the photos on their servers. Somebody needs to invent a FHE (fully homomorphic encryption) CSAM algorithm, so that Apple could scan encrypted photos for badness on the cloud.

rfd4sgmk8u 5y ago
Not Cool, now we can enforce all kinds of arbitrary censorship on encrypted data. Be careful what you wish for.
- dhosek 5y ago
  Also, the Apple system allows for a way of verifying matches to screen for false positives which the encrypted approach would not.
YPPH 5y ago
Are photos uploaded to iCloud encrypted client-side? I suggest the answer is in the negative, given the ability to look at them on the web, and the ability to reset your iCloud password using 2FA. These suggest Apple has a copy of the decryption keys, which I'm happy for them to use to scan files I have stored on their servers, for CSAM, or, within reason, anything.
- amznthrwaway 5y ago
  Yes, they’re encrypted client-side, then unencrypted in browser. This is relatively easy to verify if you do not trust an anonymous comment, and I suggest you do so.
- rovr138 5y ago
  > Are photos uploaded to iCloud encrypted client-side? Well, I can share a link to them and others can see them, including anonymous users. Just leads me to believe that either they aren’t or if they are, they have the key.
- mulmen 5y ago
  This boils down to “I have nothing to hide.” I encourage you to review https://www.schneier.com/essays/archives/2006/05/the_eternal_value_of.html https://www.schneier.com/essays/archives/2006/05/the_eternal... and consider if you really feel this way.
howaboutnope 5y ago
Alternatively, someone could realize that existing CSAM that's so widespread it's in the database basically equals water under the bridge, whereas creating CSAM that isn't in the DB requires to harm more children. Poverty and power imbalance is what harms children the most, by far. But we can't tackle that without stepping on the toes of greed, so we do circus instead.
- JKCalhoun 5y ago
  You say water under the bridge, others will say where there's smoke there's fire.
  - howaboutnope 5y ago
    But this isn't smoke, this is a photo of smoke. And as I said, the raging fires all of us can see in plain view go mostly unchecked. Where there is a photo of smoke, there may be fire, but where there is obviously a huge blazing fire, there surely is fire.
oconnore 5y ago
In some ways, I think privacy advocates could be shooting themselves in the foot here. I think that mainstream storage encryption (mainstream as in WhatsApp and Signal, as opposed to encrypting your NAS using linux commands) is very important for privacy, and that deploying it without running into CSAM objections and legislation is a Sisyphean task for for-profit businesses. Messaging seems to be more legally defensible than storage from a business perspective. If you take as an axiom that mainstream businesses will be forced to protect themselves against contributing to CSAM distribution, then the choice to offer encrypted cloud storage to non-technical end users REQUIRES doing the scan on a trusted computer (Apple has chosen the phone itself). I think the arguments that this can be abused are very real, but it's worth talking about how to fix that, because I think the alternative might be sacrificing E2EE cloud storage in the mainstream (as has happened with every other mainstream company). Perhaps more thought should be put into making this process auditable by the device owner (or by a trusted 3rd party -- say the EFF). Or perhaps the scanning could be federated -- say I don't want Apple doing that, but I might trust a privacy oriented non-profit to "certify" to Apple that my personal photo album is CSAM-free. Can that 3rd party scan be blinded, such that I send data that is representative of my images, but I've already anonymized my photos using a transformation? Could we audit (similar to certificate transparency): 1) What data from the device is being scanned? What data is being uploaded? 2) What "hashes" are being matched against, and how are those changing over time? Can the data lineage of the NCMEC database be audited? Would that pick up malicious hashes injected into the database? Generally, I think our privacy paradigm needs to be built in such a way that it can actually be deployed in our policy environment. More realpolitik, less ethical grandstanding.
anonuser123456 5y ago
FHE doesn't let the other party access the results of algorithms run on your data. Other parties can run an algorithm on your encrypted data, but only you can interpret the results.
- jnwatson 5y ago
  That's a good point. The device with the key would still have to forward the results.

rdudek 5y ago

This is correct. I have no problems of them scanning my icloud mail or photos. I just don't like the idea of there being a system level scanner for a hash database on my device assuming that I'm guilty until proven innocent. I was going to upgrade my iPhone this fall to latest model, now I'm very hesitant to do so and looking at alternatives.

istingray 5y ago
You might find this helpful in your search -- HN Discussion of open source phones: https://news.ycombinator.com/item?id=28164208 https://news.ycombinator.com/item?id=28164208
robertoandred 5y ago
But you do like the idea of a server-level scanner for a hash database assuming that you're guilty until proven innocent?
- smoldesu 5y ago
  Nobody does, but that's the current status-quo anyways. Almost every cloud-based service provider will do this anyways, Apple's only change is that they've moved their surveillance from their computers to mine. If I can't know how secure my phone is at all times, then I frankly can't feel comfortable using it.
- JKCalhoun 5y ago
  Yeah, I don't get the cloud/device distinction people are making. Increasingly they are one and the same (if you know what I mean).
  - ramses0 5y ago
    ODINN => On Device, Intrusive, Neural Network. You cannot photocopy a dollar bill. Kinda makes sense. https://en.wikipedia.org/wiki/EURion_constellation https://en.wikipedia.org/wiki/EURion_constellation Now imagine a "live" neural network running on your phone or camera that prevents you from taking pictures of the Eiffel Tower at night due to "Suspected Copyright Abuse". https://www.snopes.com/fact-check/photographs-of-eiffel-tower-at-night/ https://www.snopes.com/fact-check/photographs-of-eiffel-towe... Next imagine that a local government issues a digital "All Points Bulletin" which scans all pictures on all phones for a "person of interest in a crime (against the state)", conveniently associated with the emergency alert broadcast network. First they came: https://en.wikipedia.org/wiki/First_they_came_ https://en.wikipedia.org/wiki/First_they_came_...

joshstrange 5y ago

Genuine question. I know this is not the situation we find ourselves in but, if your options were on-device scanning for CSAM+encrypted iCloud Photos (that Apple could not decode) OR cloud-based scanning of iCloud Photos which would you pick? Are you still opposed to all on-device scanning? Edit: Thank you all for your replies. I don't share the exact same concerns as some of you but I appreciate you sharing your thoughts/reasoning. I really did what to know what people thought about this in the context of an E2E iCloud-* and it's been hard to pick that out of all the other discussions.

YPPH 5y ago
I'd pick the latter. I doubt iCloud will ever be a zero-knowledge service.
smoldesu 5y ago
Probably the latter, because frankly E2E encryption means very little to me when one end is the world's most powerful company.
haaserd 5y ago
I'd happily pick cloud based scanning. At least that way I can opt out by not using their cloud. I have serious doubts that choosing not to use iCloud will actually stop Apple from running scanning code on my device. Having a capability pretty much ensures that it will be used, and likely misused.
echelon 5y ago
My house shouldn't be full of eyes and ears. CSAM today, political materials another. This could also be used to identify whistle blowers, reporters' sources, and more. I'd hate to have gay porn on my iPhone in a Sharia law state. Or images of tank man in mainland China. Imagine when the detector extends to not just files. Things typed or said. This is a steep cliff, and we're drawing closer to the edge.
- zepto 5y ago
  > Or images of tank man in mainland China. The Chinese government forcibly installs spyware on people’s phones today. What Apple does or doesn’t do is of no consequence to them.

deleted 5y ago

[deleted]

throwawaymanbot 5y ago

The surveillance infrastructure IS the point. NO point just doing it in the cloud. It HAS to be on device.

c7DJTLrn 5y ago

You've hit the nail on the head for me. The part that offends me is that the scanning is happening on my device that I paid for, and had no knowledge of the eventual introduction of when I bought the phone. If they want to scan cloud storage, by all means. But I shouldn't have to have my device bogged down with this extra pointless computation.

zepto 5y ago
The idea that it’s going to slow your device down is an odd one.

Infernal 5y ago

> Presumably, a trivial software update down the line could expand its ambit to locally stored files. And backing up one level, this is why I’m finally working hard to take control of my devices and data from companies like Apple. This most recent episode shows that “a trivial software update” such as the one in iOS 14.3 can introduce this on-device scanning where non existed before. We knew it was possible of course, but now we have a clear cut example of it. They’ve lost my trust and that’s that.

MrWiffles 5y ago
> [...] They’ve lost my trust [...] Exactly how I feel. I used to cautiously kinda trust Apple's so-called "commitment" to privacy, but this change just shows us that any such promises are purely theater. Now that the damage is done, I doubt it can be undone, especially in my case. My opinion on Apple's credibility has crashed through the floor. In my opinion, they're now as untrustworthy as Google or Facebook.
- istingray 5y ago
  For me it's a step further. I cannot trust any company for more than privacy theater. Apple was simply the last to fall. I'm starting to look at companies to see if they meet these must-haves: 1. E2E encryption 2. Open source hardware and software Everything else is "nice to have". The current options out there are missing a lot of "nice to haves", but they will get better. I'll be a vocal customer telling them what I want.
mbjdesign 5y ago
“Trust” is entirely my issue with Apple too. I spent some time - and even wrote a post on my site - trying to clarify my thoughts and the steady erosion of trust seemed to be what I most object to. I too am trying to take control and responsibility for the computers I use now. Proving tricky, but baby steps…
- istingray 5y ago
  Care to share the link to the post on your site?

zepto 5y ago

> Presumably, a trivial software update down the line could expand its ambit to locally stored files. Sure but this is equally true now as it will be after they deploy the CSAM detector. It’s not a real argument against the detector.

skoskie 5y ago

Doing so would sacrifice privacy. I'll take the hit on my battery.

_trampeltier 5y ago

I asked in another thread (and got downvoted), if Apple does it on the user devices to save money. How much would it cost to scan everything on the serverside?

mulmen 5y ago

Ideally they wouldn’t be able to scan my iCloud photos at all. There’s real value in being able to maintain privacy while also gaining the benefits of live, remote storage and backup. The generous interpretation is that Apple moved this on-device to preserve user privacy. They just miscalculated the response to an admitted backdoor.

nitrogen 5y ago

My problem is the use of my own device to run the scan. It's a waste of system resources. I'll restate what I posted on another thread about this [0]. There should be a clear, bright line here. ---- In all these threads everyone is coming close to the crux of the issue, but I want to restate it in clearer terms: There is a sacrosanct line between "public" and "private," "mine" and "yours." That line cannot be crossed by Western governments without a warrant. Cloud computing has deliberately blurred this line over time. This on-device scanning implementation blows right past that line. Our tools before the computing revolution, and our devices after, become a part of us. Our proprioception extends to include them as part of "self." A personal device -- a tool that should be wholly owned and wholly dependable, like pen and paper -- that betrays its user, is a self that betrays itself. [0]: https://news.ycombinator.com/item?id=28162412 https://news.ycombinator.com/item?id=28162412

nostromo 5y ago
You're exactly right. We have a word for what Apple is installing on your device: spyware. And it's worse than typical spyware in that it's using your device to spy on you and report its findings back to law enforcement.
Terretta 5y ago
Framing as “on device scanning” is contrary to facts. It’s the client SDK operating the same algo on an upload event image at the client end of the upload API as it would run at the server end, just not having to decrypt it to do it. But to your theme — there’s no clear bright line in client SDKs. Devs do not think of these or handle these as if they belong to the user. If you study the cesspool of user tracking and other privacy violations on iOS devices, it’s mostly developers using the same client-side SDKs across their apps, and users none the wiser. Sometimes devs also have no idea. At least, they profess surprise and drop these SDKs when the user behavior harvesting is pointed out to them. For whatever reason, everyone’s OK with this. Perhaps the line is too blurred between client and server, thanks to emergence of concepts like node.js (“run same code on client or server”) that make this sound like a feature. Or maybe it really shouldn’t matter where the code runs provided it respects boundaries, or better, those boundaries of intentionality and access are technically guaranteed inviolate.

wayneftw 5y ago

This is exactly what I complained about in past threads and nearly nobody agreed with me there. Seems like the tide has changed! This is my only complaint. I don't care what you do on your own cloud, don't try and force my device to do the heavy lifting especially when the activity is not in my own interest.