3 ms·

LEO is not alerted automatically, where’d you get that idea?

hypothesis 5y ago
They will be if you collide a low-res image that resembles CSAM. Why would person doing manual review risk his job in case if he’s unsure? Naturally he will just play it safe and report images.
- ec109685 5y ago
  Not resembles. The adversarial image has to match a private perceptual hash function of the same CSAM image that the NeuralHash function matched before a human reviewer ever looks at it.
  - onepunchedman 5y ago
    Do you have any material on this private function?
    - ec109685 5y ago
      Not beyond the documents Apple has shared. Presumably it will be kept that way given it prevents an adversarial attack against it.
silisili 5y ago
They'd more or less have to be. Well, not necessarily 'police', but NCMEC. I did work in automating abuse detection years back, and the US govt clearly tells you are not to open/confirm suspected, reported, or happened upon cp. There's a lot of other seemingly weird laws and rules around it.
- robertoandred 5y ago
  Those laws don’t apply if it’s part of the reporting process. Apple’s stated that they do a manual to decide whether to send a report to NCMEC or not, just like other companies do.
  - silisili 5y ago
    Of course they do. If they didn't, every seedy pedo would be in the process of making a "report." It's probably also why Apple is using 'visual derivatives' for confirmation, rather than the image, though I can't find info on exactly how low resolution 'visual derivatives' are. It is of course possible that companies may get some special sign off from LE/NCMEC to do this kind of work - I won't argue with you on that as I truly don't know. I can just tell you my company did not, and was very harshly told how to proceed despite knowing the nature of what we were trying to accomplish. But, we weren't anywhere near Apple big. I remember chatting with our legal team, who made it explicit that laws didn't to cover carve outs - basically 'seeing' was illegal. But as you can imagine, police didn't come busting down our doors for happening upon it and reporting it. If you have links to law where this is not the case, I'll gladly eat crow. I've never looked myself and relied on what the lawyers had said.

They will be if you collide a low-res image that resembles CSAM. Why would person doing manual review risk his job in case if he’s unsure? Naturally he will just play it safe and report images.

ec109685 5y ago
Not resembles. The adversarial image has to match a private perceptual hash function of the same CSAM image that the NeuralHash function matched before a human reviewer ever looks at it.
- onepunchedman 5y ago
  Do you have any material on this private function?
  - ec109685 5y ago
    Not beyond the documents Apple has shared. Presumably it will be kept that way given it prevents an adversarial attack against it.

silisili 5y ago

They'd more or less have to be. Well, not necessarily 'police', but NCMEC. I did work in automating abuse detection years back, and the US govt clearly tells you are not to open/confirm suspected, reported, or happened upon cp. There's a lot of other seemingly weird laws and rules around it.

robertoandred 5y ago
Those laws don’t apply if it’s part of the reporting process. Apple’s stated that they do a manual to decide whether to send a report to NCMEC or not, just like other companies do.
- silisili 5y ago
  Of course they do. If they didn't, every seedy pedo would be in the process of making a "report." It's probably also why Apple is using 'visual derivatives' for confirmation, rather than the image, though I can't find info on exactly how low resolution 'visual derivatives' are. It is of course possible that companies may get some special sign off from LE/NCMEC to do this kind of work - I won't argue with you on that as I truly don't know. I can just tell you my company did not, and was very harshly told how to proceed despite knowing the nature of what we were trying to accomplish. But, we weren't anywhere near Apple big. I remember chatting with our legal team, who made it explicit that laws didn't to cover carve outs - basically 'seeing' was illegal. But as you can imagine, police didn't come busting down our doors for happening upon it and reporting it. If you have links to law where this is not the case, I'll gladly eat crow. I've never looked myself and relied on what the lawyers had said.