4 ms·
> how big the number of false positives would have to be to shut the whole thing down? Since it seems that false positives are uncorrectable (given that even a
by josephcsible 4y ago
> how big the number of false positives would have to be to shut the whole thing down?
Since it seems that false positives are uncorrectable (given that even a New York Times article hasn't gotten this one fixed), and the consequences of one are thus life-destroying, even a single false positive should mean shutting the whole thing down.
- tomrod 4y agoI disagree. The value Google and similar web services brings is enormous. That said, there should be people in the loop who are trained and authorized to work around the operational systems. This is the cost of doing business.
- josephcsible 4y agoI don't mean shutting down all of Google. I mean shutting down the whole automatic image scanning and account banning system.
- sulam 4y agoSo, Google, Apple, Twitter, Facebook, etc should all shut down their scanning for CSAM? Do you realize the outcry this would produce? Google is not unique here, nor does it have unique power. This story would be just as plausible, and difficult, if the company in question were Facebook or Apple. Once a system exists to scan for this material, it’s quite easy to argue that shutting it off would be immoral and potentially criminal. “You had the means to identify abused children and help them, but you turned it off because you couldn’t work out a reasonable policy to deal with the false positives? Try harder, people.” (Note that I do not speak for my employer here.)
- Semaphor 4y ago> So, Google, Apple, Twitter, Facebook, etc should all shut down their scanning for CSAM? What they should shut down, is their irrevocable deletion of accounts. When the false positives trigger nuclear bombs, then yes, a single false positive is too much. As far as I know, only Google does this, so everyone else can keep scanning in this argument.
- count 4y agoShutting down this new type of scanning is not the same as no longer scanning for CSAM. It's curious how the big providers have been scanning for CSAM for YEARS with nothing making the news...because hashes are much different and don't false positive like this.
- josephcsible 4y agoIt's immoral not to violate everyone's privacy, because some people are criminals?
- karaterobot 4y agoThe process that has led to our phones scanning everything we do and condemning us in an opaque extrajudicial process began with much smaller violations of privacy and trust — analysis of our emails, tracking our movement with cookies, and so on. When these practices were introduced, some people complained, saying they were part of a motion that could only ratchet in one direction: toward ubiquitous surveillance and corporate control over more and more of our lives. Cut to just a few years later, and here we are: "shutting it off would be immoral and potentially criminal". We can't go back! So, that clicking sound you hear is the ratchet turning.
- dont__panic 4y agoIt is interesting to me that CSAM is used as justification for such heavy-handed consequences and widespread surveillance. People use the internet to do all kinds of awful things, plenty of them crimes, plenty of those crimes being violent or personal property crimes. Murder, robbery, burglary, etc. Obviously CSAM is vile and not something your average citizen condones. But so are those other crimes. I suppose children "pull on the heartstrings" more than your average victim, but children are also the victims of those other crimes. Why haven't we started surveilling personal messages to detect criminal planning in advance? Because more people understand the nasty implications of the government reading your messages, but few understand the implications of machine learning "detecting" CSAM in your Google Drive or Photos?
- vineyardmike 4y agoYour broader point stands, but in the US at least “pre crime” monitoring or police intervention is a massive can of illegal worms. The planning of a crime can be illegal itself, eg “conspiracy to commit murder” is illegal, but generally it’s fraught with existing legal minefields.
- matthewdgreen 4y agoIn the UK they also search for “terrorism related content” and other types of criminal media. While this isn’t required in the US (and neither is CSAM scanning) part of the impetus for building these systems is to satisfy non-US governments that don’t have the same constitutional protections that the US does. US law enforcement officials routinely join with their non-US counterparts in urging/pressuring US firms to implement these systems, knowing that once they’re built (non-voluntarily) for non-US compliance they will likely be turned on (“voluntarily”) for US customers as well. See e.g. this open letter to Facebook authored by US AG Barr and counterparts from the UK and Australia. https://www.justice.gov/opa/pr/attorney-general-barr-signs-letter-facebook-us-uk-and-australian-leaders-regarding-use-end https://www.justice.gov/opa/pr/attorney-general-barr-signs-l...
- ajross 4y agoThat logic seems to go both ways, though. Actual child abuse is, surely, even more "life-destroying", right? So wouldn't it make more sense to tolerate any number of false positives if we can save just one kid? I'm always amazed at the number of people who come to a complicated nuanced issue like this, accuse one side of having implemented an absolutist and inflexible dystopian monster, and then promptly demand an equally absolutist solution. The libertarian answer might well be that we need to tolerate CSAM in the interests of free speech, but that's not the sense of the society we live in, nor of the laws it has passed. All paths must be middle paths.