5 ms·

It was quite a thing to see Apple's huge "What Happens on iPhone, Stays on iPhone" billboards up around town, at the exact same time they were announcing the ro

by dilap 4y ago

It was quite a thing to see Apple's huge "What Happens on iPhone, Stays on iPhone" billboards up around town, at the exact same time they were announcing the rollout of a new content-side illegal-material scan-and-notify system. (I think they've since delayed the rollout of that system indefinitely, after public outcry.)

Spooky23 4y ago
Yup, and now your materials are in clear text on their service getting scanned routinely I’m sure. That particular issue was the privacy advocacy people run amok.
misnome 4y ago
To be fair - wasn't that scanning on-device, and only uploading metadata on things that you yourself were already uploading to their cloud?
- GekkePrutser 4y ago
  It was restricted to that for the time being yes. But still a big step in the wrong direction. I don't want my own phone spying on me. It's a bridge too far. Scanning on a cloud service is a very big difference. Does it matter in practice? No. But it makes me feel very different about it. That's important too.
  - smoldesu 4y ago
    Same way I feel about it. When the data is on your servers, I fully expect it to be analyzed and checked to ensure it's compliant with the host's standards. That's part of our agreement, as customer and service. When you move that software onto the device I use, now I have to be conscious of everything I interact with. It's a horrible sinking feeling that isn't easily mitigated by platitudes like "we promise not to abuse it!"
    - Spooky23 4y ago
      That’s exactly what they proposed to do.
      - smoldesu 4y ago
        I guess we have different definitions of "Apple's servers" then.
        Spooky23 4y ago
        When you sync or upload folder to iCloud, iCloud is Apple’s property. They were scanning content before it landed on Apple’s property to enable S2E crypto. Sounds like you’re taking exception with the explicit parental notification, where a parent with a child enrolled in their iCloud “family” can request to be alerted when their minor child takes an action on the phone owned by the parent. The EFF wrote an awful blog that deliberately confused the already confusing release from Apple. Your privacy is almost certainly weaker as a result, as various entities can use a subpoena or warrant to access your files.
        falcolas 4y ago
        The problem is that the data wasn't on their servers, it was just flagged to go to Apple's servers. And once you're scanning files with one flag set, nothing technologically prevents the scanning of files without that flag being set. And to quote myself from the Google Stadia brouhaha - "companies lie in PR statements" - so I have no reason to trust Apple's statement that they would never scan other files.
        JimDabell 4y ago
        > And once you're scanning files with one flag set, nothing technologically prevents the scanning of files without that flag being set. You need to read up on how the system worked, because they picked a design that made absolutely no sense if they wanted to do that. They’d have to redesign it to work in a different way if they wanted to do that.
  - scarface74 4y ago
    The phone was analyzing what you would be sending to the cloud.
    - falcolas 4y ago
      True. What stops it from analyzing other files on your phone? A policy block. It's simple to change policies (or be forced to change policies).
      - scarface74 4y ago
        What’s stopping them from doing the same with your unencrypted photos in iCloud? From a technical standpoint, at least you are protected from future policy changes if your files in the cloud are encrypted. Understand, I am playing the devils advocate role more than anything else.
        falcolas 4y ago
        Nothing. And since it's uploaded, I've agreed that it's OK for them to scan and report on them. The key point I'm trying to make is that where the data is located matters for whether Apple (or Microsoft or Redhat or whichever company) has the ability or right to read and report on that data. > at least you are protected from future policy changes if your files in the cloud are encrypted. If, and only if, that data is never synced back to your phone (which Apples does currently).
      - JimDabell 4y ago
        > What stops it from analyzing other files on your phone? A policy block. The phone would be literally incapable of determining if there were any matches. Please read up on how it was designed to work.
- dilap 4y ago
  Yes, that's fair and true -- they promised the scanning would self-limit to content being synched with iCloud. But I think that's pretty thin gruel, since now you're just a feature-flag (or even a bug) away from all content being scanned. More broadly, the entire endeavor is very much at odds w/ the sentiment expressed in their public advertising.
  - Clent 4y ago
    The point the feature is lost here. They want to scan on device so they do not have to scan it in the cloud. Because people are in full FUD mode on this, we're stuck storing photos without full end-to-end encryption because Apple has do the scanning on their side.
    - dilap 4y ago
      Well, there is another option: Apple could actually respect your privacy, support end-to-end encryption, and not scan your content at all.
      - Jcowell 4y ago
        How does this work in regards to a companies obligation (if there is one) to scan for illegal illicit material (I don’t feel like typing that term out that we all know) ?
        dilap 4y ago
        Sure, if the law requires it, then the company must do it. At that point, you are living under a rather intrusive government!
        misnome 4y ago
        I don't think there are many governments in the western world that wouldn't get to that point. The entire point of client-side scanning was to head off the possibility of a law saying they have to scan everything server-side for the exact same reasons. > Well, there is another option: Apple could actually respect your privacy, support end-to-end encryption, and not scan your content at all. This is not an option you, or Apple, think there is to choose. Peoples FUD and misrepresentation over what was happening and your snark over "yes but their advertising campaign says things stay on the phone" (which is literally true) is IMO likely actually accelerating the privacy degradation. When they are compelled by law to scan this all centrally in one place in the cloud, I guess you will be happier that it's more invasive but just happening "elsewhere"?
- NaturalPhallacy 4y ago
  Devil's advocate: "We kill people based on metadata." -Snowden
  - handedness 4y ago
    That statement was made by General Michael Hayden[0], former National Security Agency director, former Central Intelligence Agency director[1]. [0] https://youtu.be/kV2HDM86XgI?t=1072 https://youtu.be/kV2HDM86XgI?t=1072 [1] https://en.wikipedia.org/wiki/Michael_Hayden_(general) https://en.wikipedia.org/wiki/Michael_Hayden_(general)
    - NaturalPhallacy 4y ago
      Thanks for pointing that out! I'd heard it from Snowden, but didn't realize he wasn't the original source: https://twitter.com/theyesmen/status/652963715168534528 https://twitter.com/theyesmen/status/652963715168534528
- tablespoon 4y ago
  > To be fair - wasn't that scanning on-device, and only uploading metadata on things that you yourself were already uploading to their cloud? On-device scanning like that would be pointless, though. IIRC, stuff uploaded to their cloud is already accessible to Apple for server-side scanning. The controversial thing was the on-device scans would trigger some kind of upload of un-uploaded stuff to Apple for further investigation.
  - lupire 4y ago
    Please provide a source for that claim.
  - JimDabell 4y ago
    > The controversial thing was the on-device scans would trigger some kind of upload of un-uploaded stuff to Apple for further investigation. No, a load of people assumed it would do that, but it’s not possible with the proposed scheme because the device had no knowledge of any matches.
  - misnome 4y ago
    > IIRC, stuff uploaded to their cloud is already accessible to Apple for server-side scanning. The controversial thing was the on-device scans would trigger some kind of upload of un-uploaded stuff to Apple for further investigation. No, the stated behaviour was that it would be uploaded to the cloud at the same time as the encrypted image. Uploaded data was still "encrypted" "at rest" (but possible for them to decrypt, just not done routinely). The whole point of the client-side scanning was that scanning was done locally on the phone and uploaded with it, as an explicit alternative to them regularly decrypting and scanning all user data on their servers. The stated operation was; if the phone fingerprint matched a public CSAM fingerprint list, then it would be automatically decrypted and scanned with a second, private fingerprinting algorithm on the server - only if this second, private fingerprint matched would the data be flagged up for potential violation (and inspected). The alternative to this was them just regularly scanning all data in their cloud anyway. Apparently people are happier with that option.
- yakubin 4y ago
  I wouldn't trust a corporation of this size to make sure there aren't recurring bugs which cause scanning of things I have on my device, but do not upload to their servers. If the code for scanning, flagging me and reporting to authorities is on the device, then I expect bugs (intentional or not) which will trigger it, even though I don't use iCloud. Put the scanning in the cloud, then I'll be fine with it - I don't use the cloud. Also the scanning was calculating hashes based on content, not just metadata.
  - JimDabell 4y ago
    > If the code for scanning, flagging me and reporting to authorities is on the device It’s not. The device has no idea if there are any matches. Everybody is assuming how it works without actually reading how it works.
    - deleted 4y ago
      [deleted]
scarface74 4y ago
Actually that’s perfectly in line with it “staying on your iPhone” that they were proposing to do content scanning on your phone. Not that I agree with it. But it is consistent.
- dilap 4y ago
  But results about matches don't stay on the phone, which I think is clearly a violation of the statement (unless you are interpreting it in an extremely literal way).
  - scarface74 4y ago
    It only gets sent to Apple if you turned on iCloud photo syncing to send the photos to Apple. That means the alternative would be to send the photos to Apple and Apple scans the photo. Either way you send the photo to Apple and meta data gets generated about CSAM. It’s just a matter of where the data gets generated. I’m also uneasy about it happening on the phone. But honestly, by it being processed on the phone, that means it can be encrypted before it gets to Apple’s servers. I’m basically working under the assumption that scanning for CSAM is legally required.
    - matthewdgreen 4y ago
      > I’m basically working under the assumption that scanning for CSAM is legally required. It is explicitly not legally required in the US [1]. Providers are required to report "apparent CSAM" that they find on their own, but they are not compelled to search their servers or private devices for its presence. And this is the case for a very good reason: if it was mandated by US law, then prosecutions would be subject to much stronger 4th amendment review under the "state action doctrine" (i.e., the companies are searching your files without probable cause as compelled representatives of the government.) The current arrangement evades this review under the very thin fig-leaf that US providers are doing the searching on their own. [1] https://crsreports.congress.gov/product/pdf/LSB/LSB10713 https://crsreports.congress.gov/product/pdf/LSB/LSB10713
nonbirithm 4y ago
I don't think it's possible to exist as a public-facing corporate entity without some kind of content scanning mechanism. If that were possible, then every child abuser would simply move their data onto those platforms and they'd become untouchable. Even MEGA, which signals the virtues of privacy/security through the prominence of decryption keys on its UI flows, will still report illegal content to authorities and display a message saying so if content was removed for that reason. Any publicly traded company that touts perfect privacy cannot deliver what they are claiming, or they'd become the service of choice for every type of disenfranchised person - including child abusers. Apple has received much more flak than the average corporation over this issue because this fundamental impossibility of perfect privacy clashed with its own privacy signaling in a loud way, and the flurry of debate over the technical merits of the novel, widely shared on-device scanning solution caused much more scrutiny than the boring server-side scanning that has been ubiquitous for decades. But the fact is that no matter how Apple tries to approach the CSAM problem, it will ultimately have to weed out child abusers from its servers or be publicly and legally lambasted. That is what society has decided is best for the welfare of children, and as a result we will have to live with an imperfect level of privacy as provided by such entities.

by dilap 4y ago

Spooky23 4y ago

Yup, and now your materials are in clear text on their service getting scanned routinely I’m sure. That particular issue was the privacy advocacy people run amok.

misnome 4y ago

To be fair - wasn't that scanning on-device, and only uploading metadata on things that you yourself were already uploading to their cloud?

GekkePrutser 4y ago
It was restricted to that for the time being yes. But still a big step in the wrong direction. I don't want my own phone spying on me. It's a bridge too far. Scanning on a cloud service is a very big difference. Does it matter in practice? No. But it makes me feel very different about it. That's important too.
- smoldesu 4y ago
  Same way I feel about it. When the data is on your servers, I fully expect it to be analyzed and checked to ensure it's compliant with the host's standards. That's part of our agreement, as customer and service. When you move that software onto the device I use, now I have to be conscious of everything I interact with. It's a horrible sinking feeling that isn't easily mitigated by platitudes like "we promise not to abuse it!"
  - Spooky23 4y ago
    That’s exactly what they proposed to do.
    - smoldesu 4y ago
      I guess we have different definitions of "Apple's servers" then.
      - Spooky23 4y ago
        When you sync or upload folder to iCloud, iCloud is Apple’s property. They were scanning content before it landed on Apple’s property to enable S2E crypto. Sounds like you’re taking exception with the explicit parental notification, where a parent with a child enrolled in their iCloud “family” can request to be alerted when their minor child takes an action on the phone owned by the parent. The EFF wrote an awful blog that deliberately confused the already confusing release from Apple. Your privacy is almost certainly weaker as a result, as various entities can use a subpoena or warrant to access your files.
        falcolas 4y ago
        The problem is that the data wasn't on their servers, it was just flagged to go to Apple's servers. And once you're scanning files with one flag set, nothing technologically prevents the scanning of files without that flag being set. And to quote myself from the Google Stadia brouhaha - "companies lie in PR statements" - so I have no reason to trust Apple's statement that they would never scan other files.
        JimDabell 4y ago
        > And once you're scanning files with one flag set, nothing technologically prevents the scanning of files without that flag being set. You need to read up on how the system worked, because they picked a design that made absolutely no sense if they wanted to do that. They’d have to redesign it to work in a different way if they wanted to do that.
- scarface74 4y ago
  The phone was analyzing what you would be sending to the cloud.
  - falcolas 4y ago
    True. What stops it from analyzing other files on your phone? A policy block. It's simple to change policies (or be forced to change policies).
    - scarface74 4y ago
      What’s stopping them from doing the same with your unencrypted photos in iCloud? From a technical standpoint, at least you are protected from future policy changes if your files in the cloud are encrypted. Understand, I am playing the devils advocate role more than anything else.
      - falcolas 4y ago
        Nothing. And since it's uploaded, I've agreed that it's OK for them to scan and report on them. The key point I'm trying to make is that where the data is located matters for whether Apple (or Microsoft or Redhat or whichever company) has the ability or right to read and report on that data. > at least you are protected from future policy changes if your files in the cloud are encrypted. If, and only if, that data is never synced back to your phone (which Apples does currently).
    - JimDabell 4y ago
      > What stops it from analyzing other files on your phone? A policy block. The phone would be literally incapable of determining if there were any matches. Please read up on how it was designed to work.
dilap 4y ago
Yes, that's fair and true -- they promised the scanning would self-limit to content being synched with iCloud. But I think that's pretty thin gruel, since now you're just a feature-flag (or even a bug) away from all content being scanned. More broadly, the entire endeavor is very much at odds w/ the sentiment expressed in their public advertising.
- Clent 4y ago
  The point the feature is lost here. They want to scan on device so they do not have to scan it in the cloud. Because people are in full FUD mode on this, we're stuck storing photos without full end-to-end encryption because Apple has do the scanning on their side.
  - dilap 4y ago
    Well, there is another option: Apple could actually respect your privacy, support end-to-end encryption, and not scan your content at all.
    - Jcowell 4y ago
      How does this work in regards to a companies obligation (if there is one) to scan for illegal illicit material (I don’t feel like typing that term out that we all know) ?
      - dilap 4y ago
        Sure, if the law requires it, then the company must do it. At that point, you are living under a rather intrusive government!
        misnome 4y ago
        I don't think there are many governments in the western world that wouldn't get to that point. The entire point of client-side scanning was to head off the possibility of a law saying they have to scan everything server-side for the exact same reasons. > Well, there is another option: Apple could actually respect your privacy, support end-to-end encryption, and not scan your content at all. This is not an option you, or Apple, think there is to choose. Peoples FUD and misrepresentation over what was happening and your snark over "yes but their advertising campaign says things stay on the phone" (which is literally true) is IMO likely actually accelerating the privacy degradation. When they are compelled by law to scan this all centrally in one place in the cloud, I guess you will be happier that it's more invasive but just happening "elsewhere"?
NaturalPhallacy 4y ago
Devil's advocate: "We kill people based on metadata." -Snowden
- handedness 4y ago
  That statement was made by General Michael Hayden[0], former National Security Agency director, former Central Intelligence Agency director[1]. [0] https://youtu.be/kV2HDM86XgI?t=1072 https://youtu.be/kV2HDM86XgI?t=1072 [1] https://en.wikipedia.org/wiki/Michael_Hayden_(general) https://en.wikipedia.org/wiki/Michael_Hayden_(general)
  - NaturalPhallacy 4y ago
    Thanks for pointing that out! I'd heard it from Snowden, but didn't realize he wasn't the original source: https://twitter.com/theyesmen/status/652963715168534528 https://twitter.com/theyesmen/status/652963715168534528
tablespoon 4y ago
> To be fair - wasn't that scanning on-device, and only uploading metadata on things that you yourself were already uploading to their cloud? On-device scanning like that would be pointless, though. IIRC, stuff uploaded to their cloud is already accessible to Apple for server-side scanning. The controversial thing was the on-device scans would trigger some kind of upload of un-uploaded stuff to Apple for further investigation.
- lupire 4y ago
  Please provide a source for that claim.
- JimDabell 4y ago
  > The controversial thing was the on-device scans would trigger some kind of upload of un-uploaded stuff to Apple for further investigation. No, a load of people assumed it would do that, but it’s not possible with the proposed scheme because the device had no knowledge of any matches.
- misnome 4y ago
  > IIRC, stuff uploaded to their cloud is already accessible to Apple for server-side scanning. The controversial thing was the on-device scans would trigger some kind of upload of un-uploaded stuff to Apple for further investigation. No, the stated behaviour was that it would be uploaded to the cloud at the same time as the encrypted image. Uploaded data was still "encrypted" "at rest" (but possible for them to decrypt, just not done routinely). The whole point of the client-side scanning was that scanning was done locally on the phone and uploaded with it, as an explicit alternative to them regularly decrypting and scanning all user data on their servers. The stated operation was; if the phone fingerprint matched a public CSAM fingerprint list, then it would be automatically decrypted and scanned with a second, private fingerprinting algorithm on the server - only if this second, private fingerprint matched would the data be flagged up for potential violation (and inspected). The alternative to this was them just regularly scanning all data in their cloud anyway. Apparently people are happier with that option.
yakubin 4y ago
I wouldn't trust a corporation of this size to make sure there aren't recurring bugs which cause scanning of things I have on my device, but do not upload to their servers. If the code for scanning, flagging me and reporting to authorities is on the device, then I expect bugs (intentional or not) which will trigger it, even though I don't use iCloud. Put the scanning in the cloud, then I'll be fine with it - I don't use the cloud. Also the scanning was calculating hashes based on content, not just metadata.
- JimDabell 4y ago
  > If the code for scanning, flagging me and reporting to authorities is on the device It’s not. The device has no idea if there are any matches. Everybody is assuming how it works without actually reading how it works.
  - deleted 4y ago
    [deleted]

scarface74 4y ago

Actually that’s perfectly in line with it “staying on your iPhone” that they were proposing to do content scanning on your phone. Not that I agree with it. But it is consistent.

dilap 4y ago
But results about matches don't stay on the phone, which I think is clearly a violation of the statement (unless you are interpreting it in an extremely literal way).
- scarface74 4y ago
  It only gets sent to Apple if you turned on iCloud photo syncing to send the photos to Apple. That means the alternative would be to send the photos to Apple and Apple scans the photo. Either way you send the photo to Apple and meta data gets generated about CSAM. It’s just a matter of where the data gets generated. I’m also uneasy about it happening on the phone. But honestly, by it being processed on the phone, that means it can be encrypted before it gets to Apple’s servers. I’m basically working under the assumption that scanning for CSAM is legally required.
  - matthewdgreen 4y ago
    > I’m basically working under the assumption that scanning for CSAM is legally required. It is explicitly not legally required in the US [1]. Providers are required to report "apparent CSAM" that they find on their own, but they are not compelled to search their servers or private devices for its presence. And this is the case for a very good reason: if it was mandated by US law, then prosecutions would be subject to much stronger 4th amendment review under the "state action doctrine" (i.e., the companies are searching your files without probable cause as compelled representatives of the government.) The current arrangement evades this review under the very thin fig-leaf that US providers are doing the searching on their own. [1] https://crsreports.congress.gov/product/pdf/LSB/LSB10713 https://crsreports.congress.gov/product/pdf/LSB/LSB10713

nonbirithm 4y ago

I don't think it's possible to exist as a public-facing corporate entity without some kind of content scanning mechanism. If that were possible, then every child abuser would simply move their data onto those platforms and they'd become untouchable. Even MEGA, which signals the virtues of privacy/security through the prominence of decryption keys on its UI flows, will still report illegal content to authorities and display a message saying so if content was removed for that reason. Any publicly traded company that touts perfect privacy cannot deliver what they are claiming, or they'd become the service of choice for every type of disenfranchised person - including child abusers. Apple has received much more flak than the average corporation over this issue because this fundamental impossibility of perfect privacy clashed with its own privacy signaling in a loud way, and the flurry of debate over the technical merits of the novel, widely shared on-device scanning solution caused much more scrutiny than the boring server-side scanning that has been ubiquitous for decades. But the fact is that no matter how Apple tries to approach the CSAM problem, it will ultimately have to weed out child abusers from its servers or be publicly and legally lambasted. That is what society has decided is best for the welfare of children, and as a result we will have to live with an imperfect level of privacy as provided by such entities.