7 ms·

That whole incident was so misinformed. CSAM scanning takes place on the cloud with all the major players. It only has hashes for the worst of the worst stuff

That whole incident was so misinformed. CSAM scanning takes place on the cloud with all the major players. It only has hashes for the worst of the worst stuff out there. What Apple (and others do) is allow the file to be scanned unencrypted on the server. What the feature Apple wanted to add was scan the files on the device and flag anything that gets a match. That file in question would be able to be decrypted on the server and checked by a human. For everything else it was encrypted in a way it cannot be looked at. If you had icloud disabled it could do nothing. The intent was to protect data, children and reduce the amount of processing done on the server end to analyse everything. Everyone lost their mind yet it was clearly laid out in the papers Apple released on it.

cebert 2y ago
> The intent was to protect data, children and reduce the amount of processing done on the server end to analyse everything. If it’s for the children, then giving up our civil liberties is a small price to pay. I’d also like to give up liberties in the name of “terrorism”. When we willingly give up our rights out of fear, these evil people have won.
- EagnaIonat 2y ago
  > If it’s for the children, then giving up our civil liberties is a small price to pay. All your pictures and videos are currently scanned. What civil liberty did their approach change in that?
rzimmerman 2y ago
I can't believe how uninformed, angry, and still willing to argue about it people were over this. The whole point was a very reasonable compromise between a legal requirement to scan photos and keeping photos end-to-end encrypted for the user. You can say the scanning requirement is wrong, there's plenty of arguments for that. But Apple went so above and beyond to try to keep photo content private and provide E2E encryption while still trying to follow the spirit of the law. No other big tech company even bothers, and somehow Apple is the outrage target.
- ec109685 2y ago
  There isn’t a law that requires them to proactively scan photos. That is why they could turn the feature back off.
  - Scion9066 2y ago
    A law by the government requiring proactive scanning of photos would in fact make the whole situation worse in the US because there would need to be a warrant if the government is requiring the scan. As long as it's voluntary by the company and not coerced by the government, they can proactively scan.
- mrshadowgoose 2y ago
  > a legal requirement to scan photos Can you provide documentation demonstrating this requirement in the United States? It is widely understood that no such requirement exists. There's no need to compromise with any requirement, this was entirely voluntary on Apple's part. That's why people were upset. > I can't believe how uninformed Oh the irony.
  - rzimmerman 2y ago
    Should have said "potential legal requirement". There was a persistent threat of blocking the use of E2E encryption for this exact reason.
  - EagnaIonat 2y ago
    > Can you provide documentation demonstrating this requirement in the United States? PROTECT Act of 2003 - Details CSAM materials as being illegal which is enforced by FBI + ICE. Also NCMEC which is a non-profit created by the US government that actively works in this area.
- nullc 2y ago
  > a legal requirement to scan photos There is absolutely no such legal requirement. If there were one it would constitute an unlawful search. The reason the provider scanning is lawful at all is because the provider has inspected material voluntarily handed over to them, and through their own lawful access to the customer material has independently and without the direction of the government discovered what they believe to be unlawful material. The cryptographic functionality in Apple's system was not there to protect the user's prviacy, the cryptographic function instead protected apple and their datasources from accountability by concealing the fingerprints that would cause user's private data to be exposed.
deleted 2y ago
[deleted]
mrshadowgoose 2y ago
> What the feature Apple wanted to add was scan the files on the device and flag anything that gets a match. This is not the revelation you think it is. Critics understood this perfectly. People simply did not want their devices scanning their content against some opaque uninspectable government-controlled list that might send you to jail in the case of a match. More generally, people usually want their devices working for their personal interests only, and not some opaque government purpose.
- Scion9066 2y ago
  From my understanding, it didn't scan all of the files on the device, just the files that were getting uploaded to Apple's iCloud. It was set up to scan the photos on the device because the files were encrypted before they were sent to the cloud and Apple couldn't access the contents but still wanted to try to make sure that their cloud wasn't storing anything that matched various hashes for bad content. If you never uploaded those files to the cloud, the scanning wouldn't catch any files that are only local.
  - mrshadowgoose 2y ago
    Your understanding is correct, as was/is the understanding of people critical of the feature. People simply don't want their device's default state to be "silently working against you, unless you are hyperaware of everything that needs to be disabled". Attacks on this desire were felt particularly strongly due to Apple having no legal requirement to implement that functionality. One also can't make the moral argument that the "bad content" list only included CSAM material, as that list was deliberately made opaque. It was a "just trust me bro" situation.
    - EagnaIonat 2y ago
      > People simply don't want their device's default state to be "silently working against you That was the misconception of what was happening though. Nothing happens on your device. Only when it gets to the cloud. It just puts a flag on the picture in question to have the cloud scan it. Which is exactly what happens before Apple suggested it and happens now. Except it does it for all your files. > One also can't make the moral argument that the "bad content" list only included CSAM material, as that list was deliberately made opaque. It was a "just trust me bro" situation. CSAM database is run by Interpol. What evidence do you have that they are not being honest?
Hackbraten 2y ago
That technology of perceptional hashes could have failed in numerous ways, ruining lives of law-abiding users along the way.
- EagnaIonat 2y ago
  The chance of a hash colliding is near 0%. The hashes are for some of the worst content out there, its not trying to detect anything else. Even so a human is in the loop to review what got a hit. Which is exactly currently happens now.
  - Hackbraten 2y ago
    > The chance of a hash colliding is near 0% Until someone finds a successful collision attack. > Even so a human is in the loop to review what got a hit. Until shareholder/growth pressure causes them to replace that human with an AI.
    - nullc 2y ago
      > Until someone finds a successful collision attack. Indeed, and within hours of the hash function being made available to me I developed a second preimage attack (strictly stronger than a collision attack)... allowing me to modify images in a visually non-objectionable way to match an arbitrary hash value. > Until shareholder/growth pressure causes them to replace that human with an AI. Indeed, but more than that: The "human" is still a dire privacy loss. Perhaps Apple's review might have protected you from some erroneous reports to law enforcement, but does it protect you from an apple-employed stalker ex? does it protect you from paparazzi? Does it protect you from the double thinking ("do I photograph my kids playing in the sprinklers? do I take a nude photo of my spouse?") due knowing that your private activity is being watched? One could easily argue that some AI second level review is an "improvement", which is another reason why your assumption that even that backstop would eventually be removed is a reasonable one.
    - EagnaIonat 2y ago
      > Until someone finds a successful collision attack. The only way to get a successful collision attack is to have a picture/video that is stored in that hash database. To put it in perspective. You randomly picking the same atom twice in the observable universe has a higher chance than getting a mismatch. > Until shareholder/growth pressure causes them to replace that human with an AI. How do you think it will impact shareholders to know that the company is not actively scanning for illegal content. Also it's Interpol/FBI that get involved.
privacyking 2y ago
Apple sells their products in oppressive regimes which force them to implement region specific features. E.g. China has their own iCloud, presumeably so it can be easily snooped on. If they were to add this anti-CSAM feature, it is not unreasonable to think that Apple would be forced to add non-CSAM stuff to the database in these countries, e.g. anything against a local dictatorship/ etc. Adding the feature would only catch the low hanging CSAM fruit, at the cost of great privacy and probably human life. If it was going to stop CSAM once and for all, it could possibly be justified, but that's not the case.
- foldr 2y ago
  If China can force Apple to do that stuff, then it can do that regardless of whether or not they add this feature.
- EagnaIonat 2y ago
  Apple and others already scan peoples pictures/videos for this stuff, so your argument can be applied to what it is now. Apples suggestion would have meant your data would be more protected as even they would not have been able to unencrypt your data.
flik 2y ago
Yes this is better than upload the entire photo. Just like virus scan can be done entirely on device, can flagging be local?. If homeomorphic encryption allows similarity matching, does not seem entirely private. Can people be matched?
amatecha 2y ago
"It only has hashes for the worst of the worst stuff out there." [citation needed] I know someone whose MS account was permabanned because they had photos of their own kid in the bathtub. I mean, I guess the person could have been lying, but I doubt they would even have been talking about it if the truth was less innocuous.
- EagnaIonat 2y ago
  > [citation needed] It is actually detailed in Apples paper. Also: https://www.interpol.int/en/Crimes/Crimes-against-children/International-Child-Sexual-Exploitation-database https://www.interpol.int/en/Crimes/Crimes-against-children/I... It works by generating a hash on known materials. Those hashes are shared with other companies so they can find that material without having to see the horrific stuff. The chance of a hash collision was also detailed in the paper which is so low to be non-existent. Even if a clash occurs a human still reviews the materials, and it normally needs a couple of hits to trigger an audit (again according to apples paper on it). > I know someone whose MS account was permabanned because they had photos of their own kid in the bathtub So you ask me for a citation and then give me anecdotal evidence? Even if that happened it has nothing to do with CSAM.
- dwaite 2y ago
  Sure, and they do that because Microsoft's CSAM detection product (which other providers like Google supposedly use) operates by having unencrypted data access to your files in the cloud. What Apple wanted to do is do those operations using homomorphic encryption and threshold key release so that the data was checked while still encrypted, and only after having a certain number of high likelihood matches would the possibility exist to see the encrypted data. So the optimistic perspective was that it was a solid win against the current state of the industry (cloud accounts storing information unencrypted so that CSAM products can analyze data), while the pessimistic perspective was that your phone was now acting as a snitch on your behavior (slippery slope etc.)
  - amatecha 2y ago
    I'm just refuting what the person I responded to said, because apparently these services have hashes for more than just "the worst of the worst stuff" or whatever.
troupo 2y ago
> Everyone lost their mind yet it was clearly laid out in the papers Apple released on it. And people working with CSAM and databases of CSAM have said it was a very bad idea.
- EagnaIonat 2y ago
  Citation needed. As the latest news suggests the opposite.
  - troupo 2y ago
    > Citation needed. The best one I remember is this one: https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On... > As the latest news suggests the opposite. What news?
    - EagnaIonat 2y ago
      They don't work on the CSAM. They are a cloud provider who makes CSAM reports. The only thing I see that is wrong in what they claimed is this: > The problem is that you don't know which pictures will be sent to Apple. Apple said in their paper that they don't send anything that is flagged. When the cloud sync occurs the files that are flagged get reviewed. All other files remain encrypted. > What news? https://www.nytimes.com/2024/12/08/technology/apple-child-sexual-abuse-material-lawsuit.html https://www.nytimes.com/2024/12/08/technology/apple-child-se...
      - troupo 2y ago
        > They don't work on the CSAM. They are a cloud provider who makes CSAM reports. --- start quote --- If you sort NCMEC's list of reporting providers by the number of submissions in 2020, then I come in at #40 out of 168. For 2019, I'm #31 out of 148 I repeatedly begged NCMEC for a hash set so I could try to automate detection. Eventually (about a year later) they provided me with about 20,000 MD5 hashes that match known CP. In addition, I had about 3 million SHA1 and MD5 hashes from other law enforcement sources. --- end quote --- Somehow this isn't "working with CSAM or CSAM databases", oh well. > Apple said in their paper that they don't send anything that is flagged. This is literally addressed in the article you pretend you read. > https://www.nytimes.com/2024/12/08/technology/apple-child-se https://www.nytimes.com/2024/12/08/technology/apple-child-se... I don't have access to this article. How does it show that people working with CSAM say Apple's implementation is a good idea?

cebert 2y ago

> The intent was to protect data, children and reduce the amount of processing done on the server end to analyse everything. If it’s for the children, then giving up our civil liberties is a small price to pay. I’d also like to give up liberties in the name of “terrorism”. When we willingly give up our rights out of fear, these evil people have won.

EagnaIonat 2y ago
> If it’s for the children, then giving up our civil liberties is a small price to pay. All your pictures and videos are currently scanned. What civil liberty did their approach change in that?

rzimmerman 2y ago

I can't believe how uninformed, angry, and still willing to argue about it people were over this. The whole point was a very reasonable compromise between a legal requirement to scan photos and keeping photos end-to-end encrypted for the user. You can say the scanning requirement is wrong, there's plenty of arguments for that. But Apple went so above and beyond to try to keep photo content private and provide E2E encryption while still trying to follow the spirit of the law. No other big tech company even bothers, and somehow Apple is the outrage target.

ec109685 2y ago
There isn’t a law that requires them to proactively scan photos. That is why they could turn the feature back off.
- Scion9066 2y ago
  A law by the government requiring proactive scanning of photos would in fact make the whole situation worse in the US because there would need to be a warrant if the government is requiring the scan. As long as it's voluntary by the company and not coerced by the government, they can proactively scan.
mrshadowgoose 2y ago
> a legal requirement to scan photos Can you provide documentation demonstrating this requirement in the United States? It is widely understood that no such requirement exists. There's no need to compromise with any requirement, this was entirely voluntary on Apple's part. That's why people were upset. > I can't believe how uninformed Oh the irony.
- rzimmerman 2y ago
  Should have said "potential legal requirement". There was a persistent threat of blocking the use of E2E encryption for this exact reason.
- EagnaIonat 2y ago
  > Can you provide documentation demonstrating this requirement in the United States? PROTECT Act of 2003 - Details CSAM materials as being illegal which is enforced by FBI + ICE. Also NCMEC which is a non-profit created by the US government that actively works in this area.
nullc 2y ago
> a legal requirement to scan photos There is absolutely no such legal requirement. If there were one it would constitute an unlawful search. The reason the provider scanning is lawful at all is because the provider has inspected material voluntarily handed over to them, and through their own lawful access to the customer material has independently and without the direction of the government discovered what they believe to be unlawful material. The cryptographic functionality in Apple's system was not there to protect the user's prviacy, the cryptographic function instead protected apple and their datasources from accountability by concealing the fingerprints that would cause user's private data to be exposed.

deleted 2y ago

[deleted]

mrshadowgoose 2y ago

> What the feature Apple wanted to add was scan the files on the device and flag anything that gets a match. This is not the revelation you think it is. Critics understood this perfectly. People simply did not want their devices scanning their content against some opaque uninspectable government-controlled list that might send you to jail in the case of a match. More generally, people usually want their devices working for their personal interests only, and not some opaque government purpose.

Scion9066 2y ago
From my understanding, it didn't scan all of the files on the device, just the files that were getting uploaded to Apple's iCloud. It was set up to scan the photos on the device because the files were encrypted before they were sent to the cloud and Apple couldn't access the contents but still wanted to try to make sure that their cloud wasn't storing anything that matched various hashes for bad content. If you never uploaded those files to the cloud, the scanning wouldn't catch any files that are only local.
- mrshadowgoose 2y ago
  Your understanding is correct, as was/is the understanding of people critical of the feature. People simply don't want their device's default state to be "silently working against you, unless you are hyperaware of everything that needs to be disabled". Attacks on this desire were felt particularly strongly due to Apple having no legal requirement to implement that functionality. One also can't make the moral argument that the "bad content" list only included CSAM material, as that list was deliberately made opaque. It was a "just trust me bro" situation.
  - EagnaIonat 2y ago
    > People simply don't want their device's default state to be "silently working against you That was the misconception of what was happening though. Nothing happens on your device. Only when it gets to the cloud. It just puts a flag on the picture in question to have the cloud scan it. Which is exactly what happens before Apple suggested it and happens now. Except it does it for all your files. > One also can't make the moral argument that the "bad content" list only included CSAM material, as that list was deliberately made opaque. It was a "just trust me bro" situation. CSAM database is run by Interpol. What evidence do you have that they are not being honest?

Hackbraten 2y ago

That technology of perceptional hashes could have failed in numerous ways, ruining lives of law-abiding users along the way.

EagnaIonat 2y ago
The chance of a hash colliding is near 0%. The hashes are for some of the worst content out there, its not trying to detect anything else. Even so a human is in the loop to review what got a hit. Which is exactly currently happens now.
- Hackbraten 2y ago
  > The chance of a hash colliding is near 0% Until someone finds a successful collision attack. > Even so a human is in the loop to review what got a hit. Until shareholder/growth pressure causes them to replace that human with an AI.
  - nullc 2y ago
    > Until someone finds a successful collision attack. Indeed, and within hours of the hash function being made available to me I developed a second preimage attack (strictly stronger than a collision attack)... allowing me to modify images in a visually non-objectionable way to match an arbitrary hash value. > Until shareholder/growth pressure causes them to replace that human with an AI. Indeed, but more than that: The "human" is still a dire privacy loss. Perhaps Apple's review might have protected you from some erroneous reports to law enforcement, but does it protect you from an apple-employed stalker ex? does it protect you from paparazzi? Does it protect you from the double thinking ("do I photograph my kids playing in the sprinklers? do I take a nude photo of my spouse?") due knowing that your private activity is being watched? One could easily argue that some AI second level review is an "improvement", which is another reason why your assumption that even that backstop would eventually be removed is a reasonable one.
  - EagnaIonat 2y ago
    > Until someone finds a successful collision attack. The only way to get a successful collision attack is to have a picture/video that is stored in that hash database. To put it in perspective. You randomly picking the same atom twice in the observable universe has a higher chance than getting a mismatch. > Until shareholder/growth pressure causes them to replace that human with an AI. How do you think it will impact shareholders to know that the company is not actively scanning for illegal content. Also it's Interpol/FBI that get involved.

privacyking 2y ago

Apple sells their products in oppressive regimes which force them to implement region specific features. E.g. China has their own iCloud, presumeably so it can be easily snooped on. If they were to add this anti-CSAM feature, it is not unreasonable to think that Apple would be forced to add non-CSAM stuff to the database in these countries, e.g. anything against a local dictatorship/ etc. Adding the feature would only catch the low hanging CSAM fruit, at the cost of great privacy and probably human life. If it was going to stop CSAM once and for all, it could possibly be justified, but that's not the case.

foldr 2y ago
If China can force Apple to do that stuff, then it can do that regardless of whether or not they add this feature.
EagnaIonat 2y ago
Apple and others already scan peoples pictures/videos for this stuff, so your argument can be applied to what it is now. Apples suggestion would have meant your data would be more protected as even they would not have been able to unencrypt your data.

flik 2y ago

Yes this is better than upload the entire photo. Just like virus scan can be done entirely on device, can flagging be local?. If homeomorphic encryption allows similarity matching, does not seem entirely private. Can people be matched?

amatecha 2y ago

"It only has hashes for the worst of the worst stuff out there." [citation needed] I know someone whose MS account was permabanned because they had photos of their own kid in the bathtub. I mean, I guess the person could have been lying, but I doubt they would even have been talking about it if the truth was less innocuous.

EagnaIonat 2y ago
> [citation needed] It is actually detailed in Apples paper. Also: https://www.interpol.int/en/Crimes/Crimes-against-children/International-Child-Sexual-Exploitation-database https://www.interpol.int/en/Crimes/Crimes-against-children/I... It works by generating a hash on known materials. Those hashes are shared with other companies so they can find that material without having to see the horrific stuff. The chance of a hash collision was also detailed in the paper which is so low to be non-existent. Even if a clash occurs a human still reviews the materials, and it normally needs a couple of hits to trigger an audit (again according to apples paper on it). > I know someone whose MS account was permabanned because they had photos of their own kid in the bathtub So you ask me for a citation and then give me anecdotal evidence? Even if that happened it has nothing to do with CSAM.
dwaite 2y ago
Sure, and they do that because Microsoft's CSAM detection product (which other providers like Google supposedly use) operates by having unencrypted data access to your files in the cloud. What Apple wanted to do is do those operations using homomorphic encryption and threshold key release so that the data was checked while still encrypted, and only after having a certain number of high likelihood matches would the possibility exist to see the encrypted data. So the optimistic perspective was that it was a solid win against the current state of the industry (cloud accounts storing information unencrypted so that CSAM products can analyze data), while the pessimistic perspective was that your phone was now acting as a snitch on your behavior (slippery slope etc.)
- amatecha 2y ago
  I'm just refuting what the person I responded to said, because apparently these services have hashes for more than just "the worst of the worst stuff" or whatever.

troupo 2y ago

> Everyone lost their mind yet it was clearly laid out in the papers Apple released on it. And people working with CSAM and databases of CSAM have said it was a very bad idea.

EagnaIonat 2y ago
Citation needed. As the latest news suggests the opposite.
- troupo 2y ago
  > Citation needed. The best one I remember is this one: https://www.hackerfactor.com/blog/index.php?/archives/929-One-Bad-Apple.html https://www.hackerfactor.com/blog/index.php?/archives/929-On... > As the latest news suggests the opposite. What news?
  - EagnaIonat 2y ago
    They don't work on the CSAM. They are a cloud provider who makes CSAM reports. The only thing I see that is wrong in what they claimed is this: > The problem is that you don't know which pictures will be sent to Apple. Apple said in their paper that they don't send anything that is flagged. When the cloud sync occurs the files that are flagged get reviewed. All other files remain encrypted. > What news? https://www.nytimes.com/2024/12/08/technology/apple-child-sexual-abuse-material-lawsuit.html https://www.nytimes.com/2024/12/08/technology/apple-child-se...
    - troupo 2y ago
      > They don't work on the CSAM. They are a cloud provider who makes CSAM reports. --- start quote --- If you sort NCMEC's list of reporting providers by the number of submissions in 2020, then I come in at #40 out of 168. For 2019, I'm #31 out of 148 I repeatedly begged NCMEC for a hash set so I could try to automate detection. Eventually (about a year later) they provided me with about 20,000 MD5 hashes that match known CP. In addition, I had about 3 million SHA1 and MD5 hashes from other law enforcement sources. --- end quote --- Somehow this isn't "working with CSAM or CSAM databases", oh well. > Apple said in their paper that they don't send anything that is flagged. This is literally addressed in the article you pretend you read. > https://www.nytimes.com/2024/12/08/technology/apple-child-se https://www.nytimes.com/2024/12/08/technology/apple-child-se... I don't have access to this article. How does it show that people working with CSAM say Apple's implementation is a good idea?