29 ms·

It is not about the implementation either. It is about hostile attack towards user privacy after years of advertising: What Happens on your iPhone, stays on you

by nbzso 5y ago

It is not about the implementation either. It is about hostile attack towards user privacy after years of advertising: What Happens on your iPhone, stays on your iPhone. Privacy is King. Privacy. That's iPhone.

nicce 5y ago
The most people have misunderstood how this system actually worked. I read every paper carefully, and this was very innovative system to protect your privacy IF we compare to existing CSAM solutions. Of course the best option is to not scan at all, but that is not really option. Only those pictures which were about to be uploaded iCloud, would have been scanned, and Apple would get information about image contents only if it is flagged. The phone is blackbox and it scans your files anyway all the time, sending metadata to Apple e.g. because of the spotlight feature and photo albums, or simply syncing your data to cloud. There is massive AI behind that spotlight feature. Nothing would have changed, just the different kind of information would have been flagged, but this time encrypted. The major improvement was E2EE like system for Photos. Currently they are not E2E encrypted in Cloud. They are plaintext for Apple. iOS 15 beta had also encryption for backups, but it never reached public reach after CSAM was delayed. So we lost yet another feature which would have increased the privacy. "But it happens in my device", is not really valid argument since most people don't understand what happens in their devices in the first hand. Even AV engines sends all your data to cloud, and you can't opt out in most of the cases and it is for every file.
- hulitu 5y ago
  So you are saying that Apple has access to all your data ( = no privacy ) but this "app" will improve your privacy by scanning all your files.
  - nicce 5y ago
    It is not scanning all of your files, only those which will be send to iCloud. Scanning for part of pipeline in sync process, so it even can’t scan other files. But yes, currently Apple can access all your images on the cloud, but with this change they would have been E2E encrypted, hence improving privacy.
    - Copernicron 5y ago
      > But yes, currently Apple can access all your images on the cloud, but with this change they would have been E2E encrypted, hence improving privacy. Apple said nothing at all this would be the case. It's pure speculation from people on the internet.
      - nicce 5y ago
        It was all over the spec, whole protocol was about encryption and ”Apple can’t access your images”. Otherwise whole flagging thing was pointless.
- tjpnz 5y ago
  An innovative solution to the wrong problem. What exactly does one achieve in nailing the local pedo for old content? If he's stupid enough to be uploading it to iCloud he's stupid enough to get himself busted in all manner of other ways. The real problem is of course criminals who are currently engaged in the production of CSAM. Which Apple's scheme does nothing to address.
  - nicce 5y ago
    It is essential to stop sharing those files no matter what. It might not help on getting them caugh, but those who buy these pictures or find elsewhere, might try to share them as well. They might not be tech guys, so they might try to use some familiar service. And yes, there are many people like that. iCloud would be flooded if no action is taken against sharing.
- Copernicron 5y ago
  > I read every paper carefully, and this was very innovative system to protect your privacy IF we compare to existing CSAM solutions I did the same and I agree with you with one small caveat. For someone like me, who does not use iCloud or any other cloud service, on device scanning seriously compromises my privacy. I want to be clear that I am not doing anything that is currently illegal in my home country, but that could change under a more oppressive government. I know my opinions would definitely get me in trouble in some countries if I happened to live there. > Of course the best option is to not scan at all, but that is not really option. Why isn't it an option? I know of no legal requirement anywhere that mandates Apple has to scan fro CSAM.
  - nicce 5y ago
    > I did the same and I agree with you with one small caveat. For someone like me, who does not use iCloud or any other cloud service, on device scanning seriously compromises my privacy. I want to be clear that I am not doing anything that is currently illegal in my home country, but that could change under a more oppressive government. I know my opinions would definitely get me in trouble in some countries if I happened to live there. At least in the current solution scanning was integrated part of the iCloud sync pipeline, and therefore this would not have affected you. > Why isn't it an option? I know of no legal requirement anywhere that mandates Apple has to scan fro CSAM. Apple is too big and they try to foresee politicians behavior. With "good will" solutions they can adjust the way how they scan or implement this. With regulation it would be more specific and might even restrict encryption in some cases totally. This is quite difficult topic at Apple's scale and I don't know if anybody really knows what exactly is already required and why
nbzso 5y ago
>So we lost yet another feature which would have increased the privacy. With all the respect, there is one small detail: The search criteria is provided by third party organisation (funded by DOD), without option of public oversight due to nature of "sensitive" data set. This data set would be defined by similar "organisations" per country bases. In my humble opinion this "privacy related solution" is Stasi wet-dream on steroids. Some governments liked it so much, that they wanted it to be extended.. https://9to5mac.com/2021/09/09/csam-scan-encrypted-messages/ https://9to5mac.com/2021/09/09/csam-scan-encrypted-messages/
- Retric 5y ago
  CSAM didn’t allow that, they only included hashes provided by multiple governments. The backlash was obvious from the outside, but it’s clear someone spent a lot of time building something they felt was reasonable. Even if Apple presumably just implemented the same system in iCloud, at least people became aware governments have access to such systems.
  - lttlrck 5y ago
    Reasonable and logical: it's a similar idea to a locally hosted virus scanner scanning all your documents...
  - w1nk 5y ago
    I'm not sure I can agree that it's clear someone spent a lot of time building something they felt was reasonable. The entire technical underpinnings of this solution rely on essentially neural image embeddings, which are very well known to be susceptible to all sorts of clever attacks. Notice how within a day or two of the announcement of this whole system and it's symbols, people were already finding lots of 'hash' collisions. In the space of people and places that can train and implement/serve such embeddings, these issues are pretty widely known, which makes it very non-obvious how this happened IMO. Someone that understood all of these issues seems to have directly ignored them.
    - Retric 5y ago
ibigb 5y ago
All cloud platform scan for CSAM. Apple was going to move the scanning task off their servers, where it is now, onto apple devices only as the devices upload to apple's cloud. If you use cloud services, info does not stay on your phone, does it?
- nbzso 5y ago
  After years of using Little Snitch to defend myself from Apple telemetry, it is not an option for me to "trust" any form of closed source software. I can ditch my iPhone at any time, but Apple wanted CSAM to be part of Mac OS and this is unacceptable. Anyway, I am glad that they tried. This was a wake up call with positive financial and technological outcome for me and my business.
  - nicce 5y ago
    > I can ditch my iPhone at any time, but Apple wanted CSAM to be part of Mac OS and this is unacceptable. It is already on MacOS. It sends hashes from every unknown file on ”virus protection purposes”. Only name and hash database is different. How that changes your privacy? And to be fair, they never said publicly that it would come to Mac OS.
    - nbzso 5y ago
      >These features are coming later this year in updates to iOS 15, iPadOS 15, watchOS 8, and macOS Monterey.* http://web.archive.org/web/20210807010615/https://www.apple.com/child-safety/ http://web.archive.org/web/20210807010615/https://www.apple....
zepto 5y ago
Except that you were just wrong and there was no attack. When people complained that this was over the line, Apple didn’t do it.

by nbzso 5y ago

nicce 5y ago

The most people have misunderstood how this system actually worked. I read every paper carefully, and this was very innovative system to protect your privacy IF we compare to existing CSAM solutions. Of course the best option is to not scan at all, but that is not really option. Only those pictures which were about to be uploaded iCloud, would have been scanned, and Apple would get information about image contents only if it is flagged. The phone is blackbox and it scans your files anyway all the time, sending metadata to Apple e.g. because of the spotlight feature and photo albums, or simply syncing your data to cloud. There is massive AI behind that spotlight feature. Nothing would have changed, just the different kind of information would have been flagged, but this time encrypted. The major improvement was E2EE like system for Photos. Currently they are not E2E encrypted in Cloud. They are plaintext for Apple. iOS 15 beta had also encryption for backups, but it never reached public reach after CSAM was delayed. So we lost yet another feature which would have increased the privacy. "But it happens in my device", is not really valid argument since most people don't understand what happens in their devices in the first hand. Even AV engines sends all your data to cloud, and you can't opt out in most of the cases and it is for every file.

hulitu 5y ago
So you are saying that Apple has access to all your data ( = no privacy ) but this "app" will improve your privacy by scanning all your files.
- nicce 5y ago
  It is not scanning all of your files, only those which will be send to iCloud. Scanning for part of pipeline in sync process, so it even can’t scan other files. But yes, currently Apple can access all your images on the cloud, but with this change they would have been E2E encrypted, hence improving privacy.
  - Copernicron 5y ago
    > But yes, currently Apple can access all your images on the cloud, but with this change they would have been E2E encrypted, hence improving privacy. Apple said nothing at all this would be the case. It's pure speculation from people on the internet.
    - nicce 5y ago
      It was all over the spec, whole protocol was about encryption and ”Apple can’t access your images”. Otherwise whole flagging thing was pointless.
tjpnz 5y ago
An innovative solution to the wrong problem. What exactly does one achieve in nailing the local pedo for old content? If he's stupid enough to be uploading it to iCloud he's stupid enough to get himself busted in all manner of other ways. The real problem is of course criminals who are currently engaged in the production of CSAM. Which Apple's scheme does nothing to address.
- nicce 5y ago
  It is essential to stop sharing those files no matter what. It might not help on getting them caugh, but those who buy these pictures or find elsewhere, might try to share them as well. They might not be tech guys, so they might try to use some familiar service. And yes, there are many people like that. iCloud would be flooded if no action is taken against sharing.
Copernicron 5y ago
> I read every paper carefully, and this was very innovative system to protect your privacy IF we compare to existing CSAM solutions I did the same and I agree with you with one small caveat. For someone like me, who does not use iCloud or any other cloud service, on device scanning seriously compromises my privacy. I want to be clear that I am not doing anything that is currently illegal in my home country, but that could change under a more oppressive government. I know my opinions would definitely get me in trouble in some countries if I happened to live there. > Of course the best option is to not scan at all, but that is not really option. Why isn't it an option? I know of no legal requirement anywhere that mandates Apple has to scan fro CSAM.
- nicce 5y ago
  > I did the same and I agree with you with one small caveat. For someone like me, who does not use iCloud or any other cloud service, on device scanning seriously compromises my privacy. I want to be clear that I am not doing anything that is currently illegal in my home country, but that could change under a more oppressive government. I know my opinions would definitely get me in trouble in some countries if I happened to live there. At least in the current solution scanning was integrated part of the iCloud sync pipeline, and therefore this would not have affected you. > Why isn't it an option? I know of no legal requirement anywhere that mandates Apple has to scan fro CSAM. Apple is too big and they try to foresee politicians behavior. With "good will" solutions they can adjust the way how they scan or implement this. With regulation it would be more specific and might even restrict encryption in some cases totally. This is quite difficult topic at Apple's scale and I don't know if anybody really knows what exactly is already required and why

nbzso 5y ago

>So we lost yet another feature which would have increased the privacy. With all the respect, there is one small detail: The search criteria is provided by third party organisation (funded by DOD), without option of public oversight due to nature of "sensitive" data set. This data set would be defined by similar "organisations" per country bases. In my humble opinion this "privacy related solution" is Stasi wet-dream on steroids. Some governments liked it so much, that they wanted it to be extended.. https://9to5mac.com/2021/09/09/csam-scan-encrypted-messages/ https://9to5mac.com/2021/09/09/csam-scan-encrypted-messages/

Retric 5y ago
CSAM didn’t allow that, they only included hashes provided by multiple governments. The backlash was obvious from the outside, but it’s clear someone spent a lot of time building something they felt was reasonable. Even if Apple presumably just implemented the same system in iCloud, at least people became aware governments have access to such systems.
- lttlrck 5y ago
  Reasonable and logical: it's a similar idea to a locally hosted virus scanner scanning all your documents...
- w1nk 5y ago
  I'm not sure I can agree that it's clear someone spent a lot of time building something they felt was reasonable. The entire technical underpinnings of this solution rely on essentially neural image embeddings, which are very well known to be susceptible to all sorts of clever attacks. Notice how within a day or two of the announcement of this whole system and it's symbols, people were already finding lots of 'hash' collisions. In the space of people and places that can train and implement/serve such embeddings, these issues are pretty widely known, which makes it very non-obvious how this happened IMO. Someone that understood all of these issues seems to have directly ignored them.
  - Retric 5y ago

ibigb 5y ago

All cloud platform scan for CSAM. Apple was going to move the scanning task off their servers, where it is now, onto apple devices only as the devices upload to apple's cloud. If you use cloud services, info does not stay on your phone, does it?

nbzso 5y ago
After years of using Little Snitch to defend myself from Apple telemetry, it is not an option for me to "trust" any form of closed source software. I can ditch my iPhone at any time, but Apple wanted CSAM to be part of Mac OS and this is unacceptable. Anyway, I am glad that they tried. This was a wake up call with positive financial and technological outcome for me and my business.
- nicce 5y ago
  > I can ditch my iPhone at any time, but Apple wanted CSAM to be part of Mac OS and this is unacceptable. It is already on MacOS. It sends hashes from every unknown file on ”virus protection purposes”. Only name and hash database is different. How that changes your privacy? And to be fair, they never said publicly that it would come to Mac OS.
  - nbzso 5y ago
    >These features are coming later this year in updates to iOS 15, iPadOS 15, watchOS 8, and macOS Monterey.* http://web.archive.org/web/20210807010615/https://www.apple.com/child-safety/ http://web.archive.org/web/20210807010615/https://www.apple....

zepto 5y ago

Except that you were just wrong and there was no attack. When people complained that this was over the line, Apple didn’t do it.