4 ms·

Wait a second. You’re telling me that you believe a low-wage worker reviewing the worst of the worst human depravity, is going to stand up on a soap box and de

by cdolan 5y ago

Wait a second. You’re telling me that you believe a low-wage worker reviewing the worst of the worst human depravity, is going to stand up on a soap box and defend another nameless and faceless denizen of the Earth, when the crux of the argument is basically this: “Yeah I know the person tripped the safety threshold for CSAM, but these images aren’t that bad!” It’s not reasonable to trust the human reviewer. They put themselves at risk by going against the automated system. I’d bet 95% of people in this circumstance would pass the buck to the FBI to make their determination, at which point “your” life is already ruined.

simondotau 5y ago
This is a misunderstanding of the system. It's not a classifier, it's a hash. The only images that are going to be seen by this low-wage worker are going to be: A) Images which are a correct hash match to an image already known to NCMEC or other agencies which have already been assigned CSAM category A1 (A=prepubescent, B=sex act); B) Images which are a hash collision. According to Apple, the likelihood of a collision is 1 in 1 trillion per user account. This system isn't a child detector strapped to a porn detector, being backed up by a low-wage worker making legal or editorial judgement calls. It's searching for images already known to child safety organisations—and even then only the most unambiguously horrific classification within the set of known images, far far far beyond the point where any ambiguity could possibly reside.
- deepspace 5y ago
  > This is a misunderstanding of the system. It's not a classifier, it's a hash You know that a perceptual hash has more in common with a classifier than with a cryptographic hash, right? If they were using a crypto-hash, then, yes, your argument could be valid, but with perceptual hashing your picture of your baby in the bath could VERY well generate the same hash as a CSAM image. And nobody believes Apples "1 in a trillion" number.
  - simondotau 5y ago
    I'd be surprised if a baby in the bath—or indeed any legitmately innocent photograph taken by a parent—could ever be a perceptual match to images tagged as "A1" by NCMEC and other agencies. This is the most extreme of the extreme: prepubescent minors actively engaged in sex acts.
    - farmerstan 5y ago
      I’d be surprised that a YouTube video of white noise would be flagged for a copyright violation, and yet here we are. Things may work right now in 2021. Things will always be changing. New hashes will be introduced. New code will be introduce. New laws in different countries will be introduced. Now that Apple has introduced this technology that no other phone manufacturer has, it can and will be changed to decrease privacy and increase government control. If China passes a law that states that IPhone needs to scan everyone’s phones for anti-government material, do you really think Tim Cook will say no? They already acquiesced about storing iCloud backups unencrypted on Chinese servers. And before you say that China would never do that, China and Hong Kong are hunting down the people who were videoed booing Chinese anthem in a shopping mall.
      - deleted 5y ago
        [deleted]
      - simondotau 5y ago
        > I’d be surprised that a YouTube video of white noise would be flagged for a copyright violation, and yet here we are. Speak for yourself, because that didn't surprise me at all. When your corpus of "copyrighted" material is so utterly massive and almost entirely devoid of defined rules or boundaries, this kind of error is inevitable. If anything I'm more surprised that we haven't seen even more of these kinds of matches, like the sound of generic telephone ringtones, recordings of mechanised church bells, or machinery which performs highly uniform tasks, etc. In the case of A1-categorised CSAM, the quantity of items is many orders of magnitude lower, the degree of technical curation will be orders of magnitude higher, and the thresholds for matches will be narrower. Is there a chance that the A1 corpus will have a few images that should have been classified as A2, B1 or B2? Yes. Is there a chance that it includes pictures of The Statue of Liberty or Westminster Tower? Almost certainly not.

by cdolan 5y ago

simondotau 5y ago

This is a misunderstanding of the system. It's not a classifier, it's a hash. The only images that are going to be seen by this low-wage worker are going to be: A) Images which are a correct hash match to an image already known to NCMEC or other agencies which have already been assigned CSAM category A1 (A=prepubescent, B=sex act); B) Images which are a hash collision. According to Apple, the likelihood of a collision is 1 in 1 trillion per user account. This system isn't a child detector strapped to a porn detector, being backed up by a low-wage worker making legal or editorial judgement calls. It's searching for images already known to child safety organisations—and even then only the most unambiguously horrific classification within the set of known images, far far far beyond the point where any ambiguity could possibly reside.

deepspace 5y ago
> This is a misunderstanding of the system. It's not a classifier, it's a hash You know that a perceptual hash has more in common with a classifier than with a cryptographic hash, right? If they were using a crypto-hash, then, yes, your argument could be valid, but with perceptual hashing your picture of your baby in the bath could VERY well generate the same hash as a CSAM image. And nobody believes Apples "1 in a trillion" number.
- simondotau 5y ago
  I'd be surprised if a baby in the bath—or indeed any legitmately innocent photograph taken by a parent—could ever be a perceptual match to images tagged as "A1" by NCMEC and other agencies. This is the most extreme of the extreme: prepubescent minors actively engaged in sex acts.
  - farmerstan 5y ago
    I’d be surprised that a YouTube video of white noise would be flagged for a copyright violation, and yet here we are. Things may work right now in 2021. Things will always be changing. New hashes will be introduced. New code will be introduce. New laws in different countries will be introduced. Now that Apple has introduced this technology that no other phone manufacturer has, it can and will be changed to decrease privacy and increase government control. If China passes a law that states that IPhone needs to scan everyone’s phones for anti-government material, do you really think Tim Cook will say no? They already acquiesced about storing iCloud backups unencrypted on Chinese servers. And before you say that China would never do that, China and Hong Kong are hunting down the people who were videoed booing Chinese anthem in a shopping mall.
    - deleted 5y ago
      [deleted]
    - simondotau 5y ago
      > I’d be surprised that a YouTube video of white noise would be flagged for a copyright violation, and yet here we are. Speak for yourself, because that didn't surprise me at all. When your corpus of "copyrighted" material is so utterly massive and almost entirely devoid of defined rules or boundaries, this kind of error is inevitable. If anything I'm more surprised that we haven't seen even more of these kinds of matches, like the sound of generic telephone ringtones, recordings of mechanised church bells, or machinery which performs highly uniform tasks, etc. In the case of A1-categorised CSAM, the quantity of items is many orders of magnitude lower, the degree of technical curation will be orders of magnitude higher, and the thresholds for matches will be narrower. Is there a chance that the A1 corpus will have a few images that should have been classified as A2, B1 or B2? Yes. Is there a chance that it includes pictures of The Statue of Liberty or Westminster Tower? Almost certainly not.