4 ms·

How a human looking at low res CSAM matching collision picture that looks "innocent" will be able to tell for sure it is a false positive? Can they know with ce

How a human looking at low res CSAM matching collision picture that looks "innocent" will be able to tell for sure it is a false positive? Can they know with certainty that it is a false positive and not a manipulated real image? It seems to me that they would have to report these anyway.

kemayo 5y ago
Is your argument "a human review step is fundamentally a rubber-stamp that won't reject a false-positive?" Because I don't personally think that's how it'd work out, but I'll acknowledge that I might just be an optimist. (I mean, assuming that you need ~30 matches to trigger the review phase of the process, I'd think it'd be weird to a reviewer looking for child porn if you got 30 pictures of apparently-random political documents or subversive memes.)
- intricatedetail 5y ago
  I am trying to say it's not possible to tell with certainty from a lowres picture that you are looking at false positive. For example low contrast CSAM imposed on a document could trigger NeuralHash match but the lowres image will look like a false positive.
  - kemayo 5y ago
    For your example, wouldn't that only work to make the original source image that's polluting the CSAM database look like CSAM in lowres? The actual document-image the oppressive government is looking for that'd trigger the match wouldn't have the CSAM included. That said, I do think it'd be nice to have a better demonstration of exactly what this "derivative" the reviewers would be looking at is. There's a lot of variations there, balancing false-positive privacy concerns, the mental health of the reviewers, potential downsampling issues, etc.
    - simondotau 5y ago
      I agree, it would be useful if Apple could be clearer by what they mean by a derivative. I recall reading somewhere that it's a reduced resolution, grayscale copy of the image. I can't vouch for that, but that would be a plausible notion of what the "derivative" would be. Personally I would also be placing a hard watermark in the middle of the image, or maybe some hard slashes randomly through the image, so that "clean" images cannot leak out of human review. Let's imagine that the derivative is a 0.5 megapixel, grayscale, watermarked, HEIC-compressed copy of the original image. This would be plenty to determine with zero ambiguity that the image is actually "A1" classified, i.e. depicts a prepubescent minor ("A") engaged in a sex act ("1").
- meowster 5y ago
  Police often rubber-stamp computer-generated evidence of infractions (facial-recognition that does not visually match is one very-applicable example), and content moderators are probably paid less.

kemayo 5y ago

Is your argument "a human review step is fundamentally a rubber-stamp that won't reject a false-positive?" Because I don't personally think that's how it'd work out, but I'll acknowledge that I might just be an optimist. (I mean, assuming that you need ~30 matches to trigger the review phase of the process, I'd think it'd be weird to a reviewer looking for child porn if you got 30 pictures of apparently-random political documents or subversive memes.)

intricatedetail 5y ago
I am trying to say it's not possible to tell with certainty from a lowres picture that you are looking at false positive. For example low contrast CSAM imposed on a document could trigger NeuralHash match but the lowres image will look like a false positive.
- kemayo 5y ago
  For your example, wouldn't that only work to make the original source image that's polluting the CSAM database look like CSAM in lowres? The actual document-image the oppressive government is looking for that'd trigger the match wouldn't have the CSAM included. That said, I do think it'd be nice to have a better demonstration of exactly what this "derivative" the reviewers would be looking at is. There's a lot of variations there, balancing false-positive privacy concerns, the mental health of the reviewers, potential downsampling issues, etc.
  - simondotau 5y ago
    I agree, it would be useful if Apple could be clearer by what they mean by a derivative. I recall reading somewhere that it's a reduced resolution, grayscale copy of the image. I can't vouch for that, but that would be a plausible notion of what the "derivative" would be. Personally I would also be placing a hard watermark in the middle of the image, or maybe some hard slashes randomly through the image, so that "clean" images cannot leak out of human review. Let's imagine that the derivative is a 0.5 megapixel, grayscale, watermarked, HEIC-compressed copy of the original image. This would be plenty to determine with zero ambiguity that the image is actually "A1" classified, i.e. depicts a prepubescent minor ("A") engaged in a sex act ("1").
meowster 5y ago
Police often rubber-stamp computer-generated evidence of infractions (facial-recognition that does not visually match is one very-applicable example), and content moderators are probably paid less.