2 ms·

10% of all OK images may be falsely flagged, but what about specific categories of images? What would the error rate be for an album of legal pornography? Wha

10% of all OK images may be falsely flagged, but what about specific categories of images? What would the error rate be for an album of legal pornography? What would the error rate be for an album of a family spending the day at a beach?

sparsely 4y ago
> 10% of all OK images may be falsely flagged, but what about specific categories of images? The article doesn't say what % of OK images are falsely flagged, only what percent of flagged images are OK. Agree that subsets might have different stats, but there's 0 information in the featured article about any of that!
- Retr0id 4y ago
  Ah, I see what you mean.
kmeisthax 4y ago
The (false negative) error rate on a parent taking pictures of an unusual growth on their child's naughty bits and texting it to their doctor is 0% - as in, the algorithm correctly identifies this as CSAM every time. Nevertheless, nobody[0] would consider this CSAM, based purely on context that... absolutely will not be available to any of the intermediaries charged with scanning for it. What system does the scanning won't matter, because it's not a question of accuracy; it's a question of missing information. This isn't even a hypothetical. The case I mentioned above actually happened. Google narc'd on someone who did exactly the thing I mentioned, and they got a police investigation for their trouble. And while the police - armed with exhonorating context - did eventually drop charges, they are still banned from Google for life. Yes, Google was contacted by journalists about this guy and they said they stand by their initial decision to permaban him. The core danger with any automated scanning system is not the false positive or false negative rate - that is an engineering problem. It will get better. The danger is the amount of legal-but-crime-adjacent activities that will be effectively prosecuted as crimes. I mentioned child telehealth above, but there's other instances of automated prosecution expanding far beyond the letter or spirit of the law. YouTube Content ID is top-of-mind for me; though that has two problems. It both kills fair use, and correctly prosecutes copyright infringements that people expect to fly under the radar. [0] Child anti-explotation agencies still caution against this because they don't want your kid to get used to getting photographed down there. However, they wouldn't consider this sexual abuse.

sparsely 4y ago

> 10% of all OK images may be falsely flagged, but what about specific categories of images? The article doesn't say what % of OK images are falsely flagged, only what percent of flagged images are OK. Agree that subsets might have different stats, but there's 0 information in the featured article about any of that!

Retr0id 4y ago
Ah, I see what you mean.

kmeisthax 4y ago

The (false negative) error rate on a parent taking pictures of an unusual growth on their child's naughty bits and texting it to their doctor is 0% - as in, the algorithm correctly identifies this as CSAM every time. Nevertheless, nobody[0] would consider this CSAM, based purely on context that... absolutely will not be available to any of the intermediaries charged with scanning for it. What system does the scanning won't matter, because it's not a question of accuracy; it's a question of missing information. This isn't even a hypothetical. The case I mentioned above actually happened. Google narc'd on someone who did exactly the thing I mentioned, and they got a police investigation for their trouble. And while the police - armed with exhonorating context - did eventually drop charges, they are still banned from Google for life. Yes, Google was contacted by journalists about this guy and they said they stand by their initial decision to permaban him. The core danger with any automated scanning system is not the false positive or false negative rate - that is an engineering problem. It will get better. The danger is the amount of legal-but-crime-adjacent activities that will be effectively prosecuted as crimes. I mentioned child telehealth above, but there's other instances of automated prosecution expanding far beyond the letter or spirit of the law. YouTube Content ID is top-of-mind for me; though that has two problems. It both kills fair use, and correctly prosecutes copyright infringements that people expect to fly under the radar. [0] Child anti-explotation agencies still caution against this because they don't want your kid to get used to getting photographed down there. However, they wouldn't consider this sexual abuse.