6 ms·

If The Verge's article is accurate about how/when the CSAM scanning occurs then I don't have a problem with that, sounds like they're moving the scanning from s

by mojzu 5y ago

If The Verge's article is accurate about how/when the CSAM scanning occurs then I don't have a problem with that, sounds like they're moving the scanning from server to client side, the concerns about false positives seem valid to me but I'm not sure the chance of one occurring has increased over the existing icloud scanning. Scope creep for other content scanning is definitely a possibility though so I hope people keep an eye on that I'm not a parent but the other child protection features seem like they could definitely be abused by some parents to exert control/pry into their kids private lives. It's a shame that systems have to be designed to prevent abuse by bad people but at Apple's scale it seems like they should have better answers for the concerns being raised

SquishyPanda23 5y ago
> sounds like they're moving the scanning from server to client side That is good, but unless a system like this is fully open source and runs only signed code there really aren't many protections against abuse.
- vineyardmike 5y ago
  And who audits the DB hashes? The code is the easiest part to trust.
  - eertami 5y ago
    You don't even need to sneak illegitimate hashes. Just use the next Pegasus-esque zero-day to dump a photo from the hash list on to the phone.
    - vineyardmike 5y ago
      Yeah, but adding whatever document/meme/etc that will represent the group you hate and boom you have a way to identify operatives of a politician party. eg. Anyone with a "Feel the Bern" marketing material -> arrest them under suspicion of CSAM. Search their device and find them as dissidents.
adriancr 5y ago
Today it does that. Tomorrow who knows... It would be easy to extend this to scan for 'wrongthink'. Next logical steps would be to scan for: confidential government documents, piracy, sensitive items, porn in some countries, LGBT content in countries where it's illegal, etc... (and not just on icloud backed up files, everything) This could come either via Apple selling this as a product or forced by governments...
- noduerme 5y ago
  I give it a month before every Pooh Bear meme ends up part of the hash DB.
  - cvwright 5y ago
    First it’s just CSAM Next it’s Covid misinformation Then eventually they’re coming for your Bernie memes
  - bostonsre 5y ago
    I'd guess more like 6 months, but I agree that it will be trivial for the CCP to make them fall in line by threatening to kick them out of the market. Although... maybe they already have this ability in China.
    - greesil 5y ago
      Maybe the CCP is the entire reason they're doing this.
      - theonlybutlet 5y ago
        My first thought exactly. How badly do they want to operate in the Chinese market.
mortenjorck 5y ago
> sounds like they're moving the scanning from server to client side, the concerns about false positives seem valid to me but I'm not sure the chance of one occurring has increased over the existing icloud scanning. That's the irony in this: This move arguably improves privacy by removing the requirement that images be decrypted on the server to run a check against the NCMEC database. While iCloud Photo Library is of course not E2E, in theory images should no longer have to be decrypted anywhere other than on the client under normal circumstances. And yet – by moving the check to the client, something that was once a clear distinction has been blurred. I entirely understand (and share) the discomfort around what is essentially a surveillance technology now running on hardware I own rather than on a server I connect to, even if it's the exact same software doing the exact same thing. Objectively, I see the advantage to Apple's client-side approach. Subjectively, I'm not so sure.
- josephcsible 5y ago
  If Apple has the ability to decrypt my photos on their servers, why do I care whether or not they actually do so today? Either way, the government could hand them a FISA warrant for them all tomorrow.
  - judge2020 5y ago
    If photos become E2EE, then Apple can no longer turn over said photos, while still not completely turning down their CSAM scanning obligations.
    - dustyharddrive 5y ago
      That’s an interesting way to look at it. Funny how this news can be interpreted as both signaling Apple’s interest in E2EE iCloud Photos or weaking their overall privacy stance.
      - judge2020 5y ago
        My issue with my own statement is that we have yet to see plans for E2EE Photos with this in place - if apple had laid out this as their intention on apple.com/child-safety/ it would have been clear-cut.
justinplouffe 5y ago
The CSAM scanning is still troubling because it implies your own device is running software against your own self-interest. If Apple wanted to get out of legal trouble by not hosting illegal content but still make sure iOS is working in the best legal interest of the phone's user, they'd prevent the upload of the tagged pictures and notify that they refuse to host these particular files. Right now, it seems like the phone will actively be snitching on its owner. I somehow don't have the same problem with them running the scan on their servers since it's machines they own but having the owner's own property work against them sets a bad precedent.
- tommymachine 5y ago
  And it’s a bat shit stupid business move
  - pcdoodle 5y ago
    I know, I was going to upgrade my 2016 SE to a 12 Mini. Now I'm not interested at all.
  - walterbell 5y ago
    Yes, what happens if many people refuse to upgrade to iOS 15 where this will be implemented? Will Apple have to issue security updates for iOS 14?
ksec 5y ago
If they are only doing it on iCloud what's wrong with continuing that? What was their incentive for having it on Phone?
- browningstreet 5y ago
  Been wondering this myself. It’s a huge move, and a big change in the presumptions of how their platform works. I’m heavily invested in the Apple ecosystem and it’ll take a years work to get off it. I’m thinking of the prevailing principles of whatever I do best, and one of them is, excise integrated platforms as much as possible. But most consumers will probably not care, and this road will get paved for miles to come.
- pritambaral 5y ago
  They pay for running iCloud; you pay for running your phone.
mulmen 5y ago
> It's a shame that systems have to be designed to prevent abuse by bad people but at Apple's scale it seems like they should have better answers for the concerns being raised. Well the obvious response is that these systems don't have to be designed. Child abuse is a convenient red herring to expand surveillance capabilities. Anyone opposing the capability is branded a child molester. This is the oldest trick in the book. I mean the capability to spy on your kid can easily be used to abuse them. Apple could very well end up making children's lives worse.
- sharken 5y ago
  It's similar to the German Covid contact-tracing app Luca, which German police is already using for other purposes. It seems the only way to opt-out is to get out of the Apple ecosystem. https://www.golem.de/news/hamburg-polizei-nutzt-corona-kontaktlisten-nach-straftat-2007-149482.html https://www.golem.de/news/hamburg-polizei-nutzt-corona-konta... https://www.ccc.de/de/updates/2021/luca-app-ccc-fordert-bundesnotbremse https://www.ccc.de/de/updates/2021/luca-app-ccc-fordert-bund...
  - mulmen 5y ago
    Luca isn't an Apple app is it? And I thought the system Apple developed with Google had much better privacy guarantees? Although I don't think it was ever actually deployed.
raxxorrax 5y ago
> I hope people keep an eye on that This will be the statement they increase the scope the next time. Hard to imagine that Tim Cook would have scanned Epsteins photos...

by mojzu 5y ago

SquishyPanda23 5y ago

> sounds like they're moving the scanning from server to client side That is good, but unless a system like this is fully open source and runs only signed code there really aren't many protections against abuse.

vineyardmike 5y ago
And who audits the DB hashes? The code is the easiest part to trust.
- eertami 5y ago
  You don't even need to sneak illegitimate hashes. Just use the next Pegasus-esque zero-day to dump a photo from the hash list on to the phone.
  - vineyardmike 5y ago
    Yeah, but adding whatever document/meme/etc that will represent the group you hate and boom you have a way to identify operatives of a politician party. eg. Anyone with a "Feel the Bern" marketing material -> arrest them under suspicion of CSAM. Search their device and find them as dissidents.

adriancr 5y ago

Today it does that. Tomorrow who knows... It would be easy to extend this to scan for 'wrongthink'. Next logical steps would be to scan for: confidential government documents, piracy, sensitive items, porn in some countries, LGBT content in countries where it's illegal, etc... (and not just on icloud backed up files, everything) This could come either via Apple selling this as a product or forced by governments...

noduerme 5y ago
I give it a month before every Pooh Bear meme ends up part of the hash DB.
- cvwright 5y ago
  First it’s just CSAM Next it’s Covid misinformation Then eventually they’re coming for your Bernie memes
- bostonsre 5y ago
  I'd guess more like 6 months, but I agree that it will be trivial for the CCP to make them fall in line by threatening to kick them out of the market. Although... maybe they already have this ability in China.
  - greesil 5y ago
    Maybe the CCP is the entire reason they're doing this.
    - theonlybutlet 5y ago
      My first thought exactly. How badly do they want to operate in the Chinese market.

mortenjorck 5y ago

> sounds like they're moving the scanning from server to client side, the concerns about false positives seem valid to me but I'm not sure the chance of one occurring has increased over the existing icloud scanning. That's the irony in this: This move arguably improves privacy by removing the requirement that images be decrypted on the server to run a check against the NCMEC database. While iCloud Photo Library is of course not E2E, in theory images should no longer have to be decrypted anywhere other than on the client under normal circumstances. And yet – by moving the check to the client, something that was once a clear distinction has been blurred. I entirely understand (and share) the discomfort around what is essentially a surveillance technology now running on hardware I own rather than on a server I connect to, even if it's the exact same software doing the exact same thing. Objectively, I see the advantage to Apple's client-side approach. Subjectively, I'm not so sure.

josephcsible 5y ago
If Apple has the ability to decrypt my photos on their servers, why do I care whether or not they actually do so today? Either way, the government could hand them a FISA warrant for them all tomorrow.
- judge2020 5y ago
  If photos become E2EE, then Apple can no longer turn over said photos, while still not completely turning down their CSAM scanning obligations.
  - dustyharddrive 5y ago
    That’s an interesting way to look at it. Funny how this news can be interpreted as both signaling Apple’s interest in E2EE iCloud Photos or weaking their overall privacy stance.
    - judge2020 5y ago
      My issue with my own statement is that we have yet to see plans for E2EE Photos with this in place - if apple had laid out this as their intention on apple.com/child-safety/ it would have been clear-cut.

justinplouffe 5y ago

The CSAM scanning is still troubling because it implies your own device is running software against your own self-interest. If Apple wanted to get out of legal trouble by not hosting illegal content but still make sure iOS is working in the best legal interest of the phone's user, they'd prevent the upload of the tagged pictures and notify that they refuse to host these particular files. Right now, it seems like the phone will actively be snitching on its owner. I somehow don't have the same problem with them running the scan on their servers since it's machines they own but having the owner's own property work against them sets a bad precedent.

tommymachine 5y ago
And it’s a bat shit stupid business move
- pcdoodle 5y ago
  I know, I was going to upgrade my 2016 SE to a 12 Mini. Now I'm not interested at all.
- walterbell 5y ago
  Yes, what happens if many people refuse to upgrade to iOS 15 where this will be implemented? Will Apple have to issue security updates for iOS 14?

ksec 5y ago

If they are only doing it on iCloud what's wrong with continuing that? What was their incentive for having it on Phone?

browningstreet 5y ago
Been wondering this myself. It’s a huge move, and a big change in the presumptions of how their platform works. I’m heavily invested in the Apple ecosystem and it’ll take a years work to get off it. I’m thinking of the prevailing principles of whatever I do best, and one of them is, excise integrated platforms as much as possible. But most consumers will probably not care, and this road will get paved for miles to come.
pritambaral 5y ago
They pay for running iCloud; you pay for running your phone.

mulmen 5y ago

> It's a shame that systems have to be designed to prevent abuse by bad people but at Apple's scale it seems like they should have better answers for the concerns being raised. Well the obvious response is that these systems don't have to be designed. Child abuse is a convenient red herring to expand surveillance capabilities. Anyone opposing the capability is branded a child molester. This is the oldest trick in the book. I mean the capability to spy on your kid can easily be used to abuse them. Apple could very well end up making children's lives worse.

sharken 5y ago
It's similar to the German Covid contact-tracing app Luca, which German police is already using for other purposes. It seems the only way to opt-out is to get out of the Apple ecosystem. https://www.golem.de/news/hamburg-polizei-nutzt-corona-kontaktlisten-nach-straftat-2007-149482.html https://www.golem.de/news/hamburg-polizei-nutzt-corona-konta... https://www.ccc.de/de/updates/2021/luca-app-ccc-fordert-bundesnotbremse https://www.ccc.de/de/updates/2021/luca-app-ccc-fordert-bund...
- mulmen 5y ago
  Luca isn't an Apple app is it? And I thought the system Apple developed with Google had much better privacy guarantees? Although I don't think it was ever actually deployed.

raxxorrax 5y ago

> I hope people keep an eye on that This will be the statement they increase the scope the next time. Hard to imagine that Tim Cook would have scanned Epsteins photos...