6 ms·

This is painful to read with the arrogant undertone while you don't actually know how CSAM scanning works. It's fuzzy hash, not ""AI"", based. Cloudflare uses

by mm983 5y ago

This is painful to read with the arrogant undertone while you don't actually know how CSAM scanning works. It's fuzzy hash, not ""AI"", based. Cloudflare uses it too, and last time i checked the web is still functional. https://support.cloudflare.com/hc/en-us/articles/360046106112-Understanding-the-CSAM-Scanning-Tool https://support.cloudflare.com/hc/en-us/articles/36004610611... How about we turn the tables and have the complainers suggest a solution. Because every single time an approach to targeted child porn takedowns has been suggested, such as datacenter raids which would not affect as many people, someone is screaming about their privacy. Child abusers evolve and are very happy if law enforcement doesn't.

babyshake 5y ago
So this means it is checking if you are sharing known CP images? That does seem to be much less invasive and problematic as there is likely no good reason to be sharing these images.
- mm983 5y ago
  Yes, that's exactly it. It uses a database compiled by NGOs and specialized firms comprising of file hashes matching child porn. These lists are handled by humans. Fuzzy means that it takes compression and the like into account, because even if just one pixel out of 20 thousand is different, the hash is different too. Fuzzy hash still recognizes it as the same image, so using an algorithm to alter the color etc. won't work.
  - ggreer 5y ago
    > These lists are handled by humans. That's also true for the no-fly list and the Terrorist Screening Database,[1] yet those are full of false positives. And unlike those lists, CSAM databases cannot be independently verified. To do so would require having the original images, which is illegal. 1. https://en.wikipedia.org/wiki/Terrorist_Screening_Database https://en.wikipedia.org/wiki/Terrorist_Screening_Database
    - sterlind 5y ago
      The no-fly list and the terrorist screening database aren't used in a court of law. The Confrontation Clause of the Sixth Amendment guarantees you access to all the evidence presented against you. You also don't need the original images to defend yourself, though apparently CP can be presented to a (traumatized) jury [0]. So if you're charged on the basis of a fuzzy hash matching, you'd subpoena Apple for the photo in your backup that matched, present it to the court (since it doesn't actually matter if it's CP or not to be admissible), and you win the case. 0. https://www.johntfloyd.com/the-difficulty-with-criminal-evidence/ https://www.johntfloyd.com/the-difficulty-with-criminal-evid...
- notadev 5y ago
  It’s not just checking files you are sharing, it’s also scanning files that exist on your device. The worry, or slippery slope argument, is that it’s one step away from scanning your device for other types of content, like memes critical of the government or just general wrongthink.
  - MikeDelta 5y ago
    A lot of governments would be very interested in this feature.
  - user-the-name 5y ago
    Slippery slope fallacy, not argument. https://yourlogicalfallacyis.com/slippery-slope https://yourlogicalfallacyis.com/slippery-slope
- native_samples 5y ago
  There actually are some edge cases even for matching against image blacklists. Google has experience with hitting them because it's used this type of image simhash for years (for shared cloud files at least). The definition of child porn varies around the world. These systems use the US definition. This is not entirely what you might expect. For example, in the USA the courts have decided that cartoons can be child porn even though no actual children are in the picture. Most of the world does not agree with this, meaning an image can be CP in one place but not another. Is Apple going to enforce the US definitions or the ones where the user actually lives? In the USA, photos an under-age person takes of themselves can also be considered CP. What counts as a "child" for sexual purposes also varies around the world. Some countries have a lower age of consent than other places. In some parts of the world the age of consent and the age at which a child stops being a child for CP purposes are different, meaning that a teenager can have sex legally but if they take a photo of themselves doing it, they are trafficking in CP. Finally, what is actually on these image blacklists? Hardly anyone actually knows because of the third rail nature of CP. Tech firms are often delivered image hashes, not even the images themselves, by third party 'charities' of various kinds and tech workers are - for obvious reasons - not normally given access to the actual pixels. Additionally, appeals from users are invariably ignored because people say "legal issues, it's complicated" and so everyone clams up. If FPs occur there is no way to resolve it and the people who see your appeal, if there even is one, won't be willing to actually look at the image to find out what it was. It should be obvious how much potential for abuse this hands the people who actually manage these CP databases. Literally any image can be made verboten immediately, without any recourse, and basically nobody will ever find out including the people who shut down the affected users.
- headShrinker 5y ago
  > So this means it is checking if you are sharing known CP images? No. NeuralMatch was “trained” using 200,000 CP images. “Neural“ is likely a reference to the perceptual matching that it uses. It is not a bit for bit match. Perceptual matching is a technique used for categorizing images based on characteristics and content. The algorithm will scan your library containing new information and compare it to what it understands as CP.
buran77 5y ago
Microsoft has been using PhotoDNA [0] to scan OneDrive content for quite some time. The news is that a device manufacturer is doing it on your device with local data. There's some MS research to use machine learning on metadata to identify offending material [1]. [0] https://en.wikipedia.org/wiki/PhotoDNA https://en.wikipedia.org/wiki/PhotoDNA [1] https://arxiv.org/pdf/2010.02387.pdf https://arxiv.org/pdf/2010.02387.pdf
cronix 5y ago
Not a single joke was made about child abuse. The specific subject of child abuse is irrelevant in my commentary. It was a commentary on the general category of AI, used all over, for many things, and more and more every day, but nice try. Edit: You edited out what I was referring to as I was replying.
BitwiseFool 5y ago
>"have the complainers suggest a solution" Being worried about privacy, establishing a precedent for scanning my data against a government database, and the risk of false positives with such an insanely emotionally charged crime is more than mere complaining. The onus should not be on me to justify why this shouldn't be done. This is something new and it is perfectly fine to argue against it without needing to provide an alternative. That being said, my solution is to continue to follow the process that law enforcement is currently using.
- mm983 5y ago
  >That being said, my solution is to continue to follow the process that law enforcement is currently using. If you knew how they approach this you wouldn't be satisfied either
  - BitwiseFool 5y ago
    This all boils down to the classic "Liberty vs. Security" dilemma. While I obviously want to see more prosecutions for people who commit these crimes, a value judgement must be made about what it takes to achieve that.
fallingknife 5y ago
> Child abusers evolve and are very happy if law enforcement doesn't. Yes. And now they will evolve by developing a simple system to modify pixels in images when they copy and transmit that will easily defeat this hashing system. The only effect this will have is that moral panickers like you will have got everybody's privacy invaded over your moral panic of the day.
- mm983 5y ago
  Again, you don't appear to know how this works. Look up fuzzy hash, i even mentioned it in the comment.
  - fallingknife 5y ago
    I have a hard time believing that this algorithm will be able to resist simple image manipulations while still being sensitive enough to avoid false positives.
    - midev 5y ago
      What algorithm are they using?
      - fallingknife 5y ago
        I don't know. If they made that public it would be completely ineffective.
    - diebeforei485 5y ago
      I have similar qualms with this, because this is increasing the number of photos scanned by 2-3 orders of magnitude, and the number of false positives presumably also increases correspondingly.
  - anthony_d 5y ago
    Just saying "fuzzy hash" doesn't begin to explain how this would work. There are an infinite variety of algorithms along with arbitrary tolerances configurable. "fuzzy hash" just isn't helpful no matter how many times you repeat it.
deleted 5y ago
[deleted]
heavyset_go 5y ago
> It's fuzzy hash, not ""AI"", based. Cloudflare uses it too, and last time i checked the web is still functional. If they're using fuzzy matching with perceptual hashes, then the space that false positives can exist in for each perceptual hash is huge.
diebeforei485 5y ago
It makes sense for website owners to scan what's being uploaded to their servers, but that is totally different from scanning what's stored locally on people's devices.
headShrinker 5y ago
There is no reference to fuzzy anything in any of the write ups. I don’t know how you can proclaim that this is a fuzzy hash. Where is your source?

by mm983 5y ago

babyshake 5y ago

So this means it is checking if you are sharing known CP images? That does seem to be much less invasive and problematic as there is likely no good reason to be sharing these images.

mm983 5y ago
Yes, that's exactly it. It uses a database compiled by NGOs and specialized firms comprising of file hashes matching child porn. These lists are handled by humans. Fuzzy means that it takes compression and the like into account, because even if just one pixel out of 20 thousand is different, the hash is different too. Fuzzy hash still recognizes it as the same image, so using an algorithm to alter the color etc. won't work.
- ggreer 5y ago
  > These lists are handled by humans. That's also true for the no-fly list and the Terrorist Screening Database,[1] yet those are full of false positives. And unlike those lists, CSAM databases cannot be independently verified. To do so would require having the original images, which is illegal. 1. https://en.wikipedia.org/wiki/Terrorist_Screening_Database https://en.wikipedia.org/wiki/Terrorist_Screening_Database
  - sterlind 5y ago
    The no-fly list and the terrorist screening database aren't used in a court of law. The Confrontation Clause of the Sixth Amendment guarantees you access to all the evidence presented against you. You also don't need the original images to defend yourself, though apparently CP can be presented to a (traumatized) jury [0]. So if you're charged on the basis of a fuzzy hash matching, you'd subpoena Apple for the photo in your backup that matched, present it to the court (since it doesn't actually matter if it's CP or not to be admissible), and you win the case. 0. https://www.johntfloyd.com/the-difficulty-with-criminal-evidence/ https://www.johntfloyd.com/the-difficulty-with-criminal-evid...
notadev 5y ago
It’s not just checking files you are sharing, it’s also scanning files that exist on your device. The worry, or slippery slope argument, is that it’s one step away from scanning your device for other types of content, like memes critical of the government or just general wrongthink.
- MikeDelta 5y ago
  A lot of governments would be very interested in this feature.
- user-the-name 5y ago
  Slippery slope fallacy, not argument. https://yourlogicalfallacyis.com/slippery-slope https://yourlogicalfallacyis.com/slippery-slope
native_samples 5y ago
There actually are some edge cases even for matching against image blacklists. Google has experience with hitting them because it's used this type of image simhash for years (for shared cloud files at least). The definition of child porn varies around the world. These systems use the US definition. This is not entirely what you might expect. For example, in the USA the courts have decided that cartoons can be child porn even though no actual children are in the picture. Most of the world does not agree with this, meaning an image can be CP in one place but not another. Is Apple going to enforce the US definitions or the ones where the user actually lives? In the USA, photos an under-age person takes of themselves can also be considered CP. What counts as a "child" for sexual purposes also varies around the world. Some countries have a lower age of consent than other places. In some parts of the world the age of consent and the age at which a child stops being a child for CP purposes are different, meaning that a teenager can have sex legally but if they take a photo of themselves doing it, they are trafficking in CP. Finally, what is actually on these image blacklists? Hardly anyone actually knows because of the third rail nature of CP. Tech firms are often delivered image hashes, not even the images themselves, by third party 'charities' of various kinds and tech workers are - for obvious reasons - not normally given access to the actual pixels. Additionally, appeals from users are invariably ignored because people say "legal issues, it's complicated" and so everyone clams up. If FPs occur there is no way to resolve it and the people who see your appeal, if there even is one, won't be willing to actually look at the image to find out what it was. It should be obvious how much potential for abuse this hands the people who actually manage these CP databases. Literally any image can be made verboten immediately, without any recourse, and basically nobody will ever find out including the people who shut down the affected users.
headShrinker 5y ago
> So this means it is checking if you are sharing known CP images? No. NeuralMatch was “trained” using 200,000 CP images. “Neural“ is likely a reference to the perceptual matching that it uses. It is not a bit for bit match. Perceptual matching is a technique used for categorizing images based on characteristics and content. The algorithm will scan your library containing new information and compare it to what it understands as CP.

buran77 5y ago

Microsoft has been using PhotoDNA [0] to scan OneDrive content for quite some time. The news is that a device manufacturer is doing it on your device with local data. There's some MS research to use machine learning on metadata to identify offending material [1]. [0] https://en.wikipedia.org/wiki/PhotoDNA https://en.wikipedia.org/wiki/PhotoDNA [1] https://arxiv.org/pdf/2010.02387.pdf https://arxiv.org/pdf/2010.02387.pdf

cronix 5y ago

Not a single joke was made about child abuse. The specific subject of child abuse is irrelevant in my commentary. It was a commentary on the general category of AI, used all over, for many things, and more and more every day, but nice try. Edit: You edited out what I was referring to as I was replying.

BitwiseFool 5y ago

>"have the complainers suggest a solution" Being worried about privacy, establishing a precedent for scanning my data against a government database, and the risk of false positives with such an insanely emotionally charged crime is more than mere complaining. The onus should not be on me to justify why this shouldn't be done. This is something new and it is perfectly fine to argue against it without needing to provide an alternative. That being said, my solution is to continue to follow the process that law enforcement is currently using.

mm983 5y ago
>That being said, my solution is to continue to follow the process that law enforcement is currently using. If you knew how they approach this you wouldn't be satisfied either
- BitwiseFool 5y ago
  This all boils down to the classic "Liberty vs. Security" dilemma. While I obviously want to see more prosecutions for people who commit these crimes, a value judgement must be made about what it takes to achieve that.

fallingknife 5y ago

> Child abusers evolve and are very happy if law enforcement doesn't. Yes. And now they will evolve by developing a simple system to modify pixels in images when they copy and transmit that will easily defeat this hashing system. The only effect this will have is that moral panickers like you will have got everybody's privacy invaded over your moral panic of the day.

mm983 5y ago
Again, you don't appear to know how this works. Look up fuzzy hash, i even mentioned it in the comment.
- fallingknife 5y ago
  I have a hard time believing that this algorithm will be able to resist simple image manipulations while still being sensitive enough to avoid false positives.
  - midev 5y ago
    What algorithm are they using?
    - fallingknife 5y ago
      I don't know. If they made that public it would be completely ineffective.
  - diebeforei485 5y ago
    I have similar qualms with this, because this is increasing the number of photos scanned by 2-3 orders of magnitude, and the number of false positives presumably also increases correspondingly.
- anthony_d 5y ago
  Just saying "fuzzy hash" doesn't begin to explain how this would work. There are an infinite variety of algorithms along with arbitrary tolerances configurable. "fuzzy hash" just isn't helpful no matter how many times you repeat it.

deleted 5y ago

[deleted]

heavyset_go 5y ago

> It's fuzzy hash, not ""AI"", based. Cloudflare uses it too, and last time i checked the web is still functional. If they're using fuzzy matching with perceptual hashes, then the space that false positives can exist in for each perceptual hash is huge.

diebeforei485 5y ago

It makes sense for website owners to scan what's being uploaded to their servers, but that is totally different from scanning what's stored locally on people's devices.

headShrinker 5y ago

There is no reference to fuzzy anything in any of the write ups. I don’t know how you can proclaim that this is a fuzzy hash. Where is your source?