6 ms·

And then the CSAM filter on your device reports you to some authority.

BoorishBears 4y ago
If you have things on your device that match entries in the CSAM database, yes there's a chance you're a victim of a targeted attack taking advantage of highly experimental collisions... but the odds you "accidentally generated" that content are not realistic.
- yeet_yeet_yeet 4y ago
  >not realistic The odds are zero. 1/2^256 = 0. In cryptography these odds are treated as zero until you generate close to 2^128 images. Unfortunately there's no word in natural English to describe how unlikely. The most precise is "zero".
  - criddell 4y ago
    How can you be so sure? As I understand it, the hash is of features in the image and not the image itself. Are the CSAM feature detection heuristics public?
  - ben_w 4y ago
    Are you assuming that digital images are evenly distributed over the set of all possible 256 bit vectors? Because I don't think that's a reasonable assumption. Even if image recognition was perfectly solved with no known edge cases (ha!), when an entire topic is a semantic stop-sign for most people, you can't expect the mysterious opaque box that is a guilty-enough-to-investigate detection mechanism to be something that gets rapid updates and corrections when new failure modes are discovered.
  - jerf 4y ago
    You should spend some time with an internet search engine and the term "perceptual hashing". What you're talking about is another type of hashing, which can be useful for classifying image files, but not images. The former has a very concrete definition that is specified down to the bit; the latter is a fuzzy space because it's trying to yield similar (not necessarily identical) hashes for images that humans consider similar. Much different space, much different problem, much different collision situation. Cryptographic hashing is not the only kind of hashing.
    - yeet_yeet_yeet 4y ago
      Oh wow https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni... so they essentially just use CNN output to automatically determine whether to report people to the authorities? For some reason I assumed they were just comparing the files they knew to be CSAM. Yeah that's bad. What about deepdream/CNN reversing? Couldn't a rogue apple engineer just create a innocuous looking false positive, say a cat picture, share it on Reddit, and everybody who downloads it is flagged to police for CSAM?
      - criddell 4y ago
        The CSAM flagging generally isn’t reported to police to prevent the situation you describe. Google would get the report and once some threshold is reached, a person reviews the report(s) and decides if the police are notified.
      - astrange 4y ago
        No, there are two hashes used in the Apple system, one public and neural and one hidden, the intent of both is to match specific known images and not unknown new ones, and the result of passing both hashes is a manual review and not automatic reporting. I've never seen a published attack that would actually be a problem; they all misread how the system worked. (Also, it's not reported to the police but to NCMEC, which is not a government agency. This is for 4th amendment privacy reasons.)
iceburgcrm 4y ago
Only available on the latest iphone
- deleted 4y ago
  [deleted]
- Gigachad 4y ago
  Apple ended up not implementing that iirc. While Google Photos has had it the whole time. Googles is actually worse. Apple was only going to match against known CSAM images while google has ML to identify new images which resulted in one parent being arrested for a medical image of their own child.
  - knaik94 4y ago
    I am fine with photos that are uploaded to the cloud being scanned. I do not want my own device spending energy and scanning images even before they completely leave my network or device. Google scanning my Google drive files is fine with me. Apple is much worse.
    - Gigachad 4y ago
      The apple one was only going to scan photos stored on iCloud. It scans them on your phone but if you don’t use iCloud it doesn’t scan anything. It’s a neat trick that means the Apple servers can know they aren’t storing illegal content without ever having to look at it. If it’s anything like the regular scanning iPhones do, it’s done overnight while plugged in.
      - knaik94 4y ago
        That makes the phone the scanner and by definition it's before iCloud. Apple makes it very hard to use an iPhone/iPad without keeping iCloud enabled. I don't want my phone spying on me, that sets a dangerous precedent. I don't care how many times it is scanned by Apple after I store it unencrypted on iCloud servers, but scanning without my permission on my device before it leave my device is a violation of my privacy. Apple will look at it if your phone detects something and it will inform authorities without letting you know. With other companies, once it is detected on their server, they forward your information to government authorities. Apple instead wanted to have an inhouse team to filter false positives before notifying authorities. Apple is worse in every single way.

If you have things on your device that match entries in the CSAM database, yes there's a chance you're a victim of a targeted attack taking advantage of highly experimental collisions... but the odds you "accidentally generated" that content are not realistic.

yeet_yeet_yeet 4y ago
>not realistic The odds are zero. 1/2^256 = 0. In cryptography these odds are treated as zero until you generate close to 2^128 images. Unfortunately there's no word in natural English to describe how unlikely. The most precise is "zero".
- criddell 4y ago
  How can you be so sure? As I understand it, the hash is of features in the image and not the image itself. Are the CSAM feature detection heuristics public?
- ben_w 4y ago
  Are you assuming that digital images are evenly distributed over the set of all possible 256 bit vectors? Because I don't think that's a reasonable assumption. Even if image recognition was perfectly solved with no known edge cases (ha!), when an entire topic is a semantic stop-sign for most people, you can't expect the mysterious opaque box that is a guilty-enough-to-investigate detection mechanism to be something that gets rapid updates and corrections when new failure modes are discovered.
- jerf 4y ago
  You should spend some time with an internet search engine and the term "perceptual hashing". What you're talking about is another type of hashing, which can be useful for classifying image files, but not images. The former has a very concrete definition that is specified down to the bit; the latter is a fuzzy space because it's trying to yield similar (not necessarily identical) hashes for images that humans consider similar. Much different space, much different problem, much different collision situation. Cryptographic hashing is not the only kind of hashing.
  - yeet_yeet_yeet 4y ago
    Oh wow https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni... so they essentially just use CNN output to automatically determine whether to report people to the authorities? For some reason I assumed they were just comparing the files they knew to be CSAM. Yeah that's bad. What about deepdream/CNN reversing? Couldn't a rogue apple engineer just create a innocuous looking false positive, say a cat picture, share it on Reddit, and everybody who downloads it is flagged to police for CSAM?
    - criddell 4y ago
      The CSAM flagging generally isn’t reported to police to prevent the situation you describe. Google would get the report and once some threshold is reached, a person reviews the report(s) and decides if the police are notified.
    - astrange 4y ago
      No, there are two hashes used in the Apple system, one public and neural and one hidden, the intent of both is to match specific known images and not unknown new ones, and the result of passing both hashes is a manual review and not automatic reporting. I've never seen a published attack that would actually be a problem; they all misread how the system worked. (Also, it's not reported to the police but to NCMEC, which is not a government agency. This is for 4th amendment privacy reasons.)

iceburgcrm 4y ago

Only available on the latest iphone

deleted 4y ago
[deleted]
Gigachad 4y ago
Apple ended up not implementing that iirc. While Google Photos has had it the whole time. Googles is actually worse. Apple was only going to match against known CSAM images while google has ML to identify new images which resulted in one parent being arrested for a medical image of their own child.
- knaik94 4y ago
  I am fine with photos that are uploaded to the cloud being scanned. I do not want my own device spending energy and scanning images even before they completely leave my network or device. Google scanning my Google drive files is fine with me. Apple is much worse.
  - Gigachad 4y ago
    The apple one was only going to scan photos stored on iCloud. It scans them on your phone but if you don’t use iCloud it doesn’t scan anything. It’s a neat trick that means the Apple servers can know they aren’t storing illegal content without ever having to look at it. If it’s anything like the regular scanning iPhones do, it’s done overnight while plugged in.
    - knaik94 4y ago
      That makes the phone the scanner and by definition it's before iCloud. Apple makes it very hard to use an iPhone/iPad without keeping iCloud enabled. I don't want my phone spying on me, that sets a dangerous precedent. I don't care how many times it is scanned by Apple after I store it unencrypted on iCloud servers, but scanning without my permission on my device before it leave my device is a violation of my privacy. Apple will look at it if your phone detects something and it will inform authorities without letting you know. With other companies, once it is detected on their server, they forward your information to government authorities. Apple instead wanted to have an inhouse team to filter false positives before notifying authorities. Apple is worse in every single way.