6 ms·

Many of these steps are spun as being for the users' privacy. But they also prevent the operators of the system – Apple Inc, quasi-governmental non-profits (li

by gojomo 5y ago

Many of these steps are spun as being for the users' privacy. But they also prevent the operators of the system – Apple Inc, quasi-governmental non-profits (like NCMEC), & government law-enforcement agencies – from facing accountability for their choices. No one else can confirm what content they're scanning for, or what 'threshold' or other review applies, before serious enforcement action is taken against an account. Also, Apple is quite vague about whether the claim "Apple manually reviews all reports" means Apple employees can view all triggering content, even if it was never uploaded to Apple servers. (How would that work?) It would be trivial to extend this system to look for specific phrases or names in all local communication content. How will Apple say 'no' when China & Russia want that capability? In fact, might this initial domestic rollout simply be the 1st step of a crafty multi-year process to make that sort of eventual request seem ho-hum, "we're just extending American law-enforcement tech to follow local laws"?

phkahler 5y ago
>> It would be trivial to extend this system to look for specific phrases or names in all local communication content. How? They're comparing image hashes.
- gojomo 5y ago
  Perceptual image hashes are a hard problem. Searching for subversive text strings is easy. And, a bunch of the obscuring-of-surveillance-intent achieved here is oblivious as to the types of detection-events - not at all image-specific. (That is, the private-set-intersection, cryptographic 'blinding', & threshold sharing schemes will work just fine for text/text-shingle/text-vector snitchlists as for image-perceptual-hash snitchlists. They're generic tricks for hiding what searches are being performed, & easy to repurpose.) Once Apple's shown a willingness to install iOS software that does a crime-grep on private data, & snitches on users without their assent, adding text-triggers is technically simple. A iOS point-release.
sirn 5y ago
> Also, Apple is quite vague about whether the claim "Apple manually reviews all reports" means Apple employees can view all triggering content, even if it was never uploaded to Apple servers. (How would that work?) From a quick read of the PDF, my impression is this Safety Voucher is more of an additional metadata generated before a photo is uploaded to iCloud Photo (as oppose to scan photo library to see if it matches and notify Apple narrative). The Safety Voucher is designed in a way that they could decide on the server-side if a photo is CSAM without requiring iCloud to have an ability to decrypt all photos. Safety Voucher comes in three layers: - The first layer is generating a NeuralHash for a photo and derive a cryptographic header from it based on blind hash provisioned by Apple from a known CSAM NeuralHash (e.g. Apple pre-compute this table for each device). First layer is decryptable if the generated NeuralHash is a known CSAM. (Is it going to be a random data if NeuralHash doesn't exists in the table?) - The second layer uses Secret Sharing[1] scheme (where a content is decryptable only if >n keys are present) using NeuralHash of a visual derivative as a secret share. Multiple visual derivative is generated for each photo, and certain amount of shares is required to decrypt the layer. - After the second layer is decrypted, the file is readable. One thing that doesn't makes sense is iCloud Photos are not currently End-to-End Encrypted, so Apple already have access to all photos. This scheme would only makes sense if they plan to introduce End-to-End Encryption for iCloud Photos (& iCloud Backups?) in the future. In my opinion, if they were to enable E2EE for photos, this scheme would be slightly better for privacy than Apple having an ability to scan all photos on the cloud (e.g. they can't retroactively decrypt a photo that already has Safety Voucher uploaded to iCloud if the matching NeuralHash doesn't exists at the time of blind hash table generation), but significantly less than having E2EE and not having this system in the first place. (As someone from a country that is known to abuse this sort of power, I'm not happy about this system existing.) [1]: https://en.wikipedia.org/wiki/Secret_sharing https://en.wikipedia.org/wiki/Secret_sharing
- gojomo 5y ago
  Your interpretation might be right, but Apple's vagueness might be strategically hiding things they know would be even more unpopular, if revealed. Still, there's the question of what additional assurance this "manual review" is going to provide, if the Apple personnel only have exactly the same metadata/scrambled-representations as the software has, and not the raw images. They couldn't detect a false positive with that info. Perhaps it's just an option for Apple executives to exercise an extra level of courtesy (or leverage) over the famous & powerful, after seeing the alarm plus the account identity? "We've got a Senator here, we better consult Tim before reporting this through normal channels."

by gojomo 5y ago

phkahler 5y ago

>> It would be trivial to extend this system to look for specific phrases or names in all local communication content. How? They're comparing image hashes.

gojomo 5y ago
Perceptual image hashes are a hard problem. Searching for subversive text strings is easy. And, a bunch of the obscuring-of-surveillance-intent achieved here is oblivious as to the types of detection-events - not at all image-specific. (That is, the private-set-intersection, cryptographic 'blinding', & threshold sharing schemes will work just fine for text/text-shingle/text-vector snitchlists as for image-perceptual-hash snitchlists. They're generic tricks for hiding what searches are being performed, & easy to repurpose.) Once Apple's shown a willingness to install iOS software that does a crime-grep on private data, & snitches on users without their assent, adding text-triggers is technically simple. A iOS point-release.

sirn 5y ago

> Also, Apple is quite vague about whether the claim "Apple manually reviews all reports" means Apple employees can view all triggering content, even if it was never uploaded to Apple servers. (How would that work?) From a quick read of the PDF, my impression is this Safety Voucher is more of an additional metadata generated before a photo is uploaded to iCloud Photo (as oppose to scan photo library to see if it matches and notify Apple narrative). The Safety Voucher is designed in a way that they could decide on the server-side if a photo is CSAM without requiring iCloud to have an ability to decrypt all photos. Safety Voucher comes in three layers: - The first layer is generating a NeuralHash for a photo and derive a cryptographic header from it based on blind hash provisioned by Apple from a known CSAM NeuralHash (e.g. Apple pre-compute this table for each device). First layer is decryptable if the generated NeuralHash is a known CSAM. (Is it going to be a random data if NeuralHash doesn't exists in the table?) - The second layer uses Secret Sharing[1] scheme (where a content is decryptable only if >n keys are present) using NeuralHash of a visual derivative as a secret share. Multiple visual derivative is generated for each photo, and certain amount of shares is required to decrypt the layer. - After the second layer is decrypted, the file is readable. One thing that doesn't makes sense is iCloud Photos are not currently End-to-End Encrypted, so Apple already have access to all photos. This scheme would only makes sense if they plan to introduce End-to-End Encryption for iCloud Photos (& iCloud Backups?) in the future. In my opinion, if they were to enable E2EE for photos, this scheme would be slightly better for privacy than Apple having an ability to scan all photos on the cloud (e.g. they can't retroactively decrypt a photo that already has Safety Voucher uploaded to iCloud if the matching NeuralHash doesn't exists at the time of blind hash table generation), but significantly less than having E2EE and not having this system in the first place. (As someone from a country that is known to abuse this sort of power, I'm not happy about this system existing.) [1]: https://en.wikipedia.org/wiki/Secret_sharing https://en.wikipedia.org/wiki/Secret_sharing

gojomo 5y ago
Your interpretation might be right, but Apple's vagueness might be strategically hiding things they know would be even more unpopular, if revealed. Still, there's the question of what additional assurance this "manual review" is going to provide, if the Apple personnel only have exactly the same metadata/scrambled-representations as the software has, and not the raw images. They couldn't detect a false positive with that info. Perhaps it's just an option for Apple executives to exercise an extra level of courtesy (or leverage) over the famous & powerful, after seeing the alarm plus the account identity? "We've got a Senator here, we better consult Tim before reporting this through normal channels."