3 ms·
Microsoft, Facebook, Google and Apple have scanned data stored on their servers for CSAM for over a decade already. The difference is that Apple is moving the s
by nonbirithm 5y ago
Microsoft, Facebook, Google and Apple have scanned data stored on their servers for CSAM for over a decade already. The difference is that Apple is moving the scan on-device. Has there been any report of even a single person who's been a victim of a PhotoDNA false positive in those ten years? I'm not trying to wave away the concerns about on-device privacy, but I'd want evidence that a such significant scale of wrongful conviction is plausible as a result of Apple's change.
I can believe that a couple of false positives would inevitably occur assuming Apple has good intentions (which is not a given), but I'm not seeing how thousands could be wrongfully prosecuted unless Apple weren't using the system like they state they will. At least in the US, I'm not seeing how a conviction can be made on the basis of a perceptual hash alone without the actual CSAM. The courts would still need the actual evidence to prosecute people. Getting people arrested on a doctored meme that causes a hash collision would at most waste the court's time, and it would only damage the credibility of perceptual hashing systems in future cases. Also, thousands of PhotoDNA false positives being reported in public court cases would only cause Apple's reputation to collapse. They seem to have enough confidence that such an extreme false positive rate is not possible to the point of implementing this change. And I don't see how just moving the hashing workload to the device fundamentally changes the actual hashing mechanism and increases the chance of wrongful conviction over the current status quo of serverside scanning (assuming that it only applies to images uploaded to iCloud, which could change of course). The proper time to be outraged at the wrongful conviction problem was ten years ago, when the major tech companies started to adopt PhotoDNA.
On the other hand, if we're talking about what the CCP might do, I would completely agree.
- 7373737373 5y ago> I'm not seeing how a conviction can be made on the basis of a perceptual hash alone without the actual CSAM This is a good point, but it's not just about people getting wrongly convicted, this system even introducing a remote possibility of having strangers view your personal files is disturbing. In the US, it violates the 4th amendment against unreasonable search, a company being the middleman doesn't change that. Privacy is a shield of the individual, here the presumption of innocence is deposed even before the trial. An extremely low false positive rate or the perceived harmlessness of the current government don't matter, the systems' existence is inherently wrong. It's an extension of the warrantless surveillance culture modern nations are already so good at. "It is better that ten guilty persons escape than that one innocent suffer." - https://en.wikipedia.org/wiki/Blackstone%27s_ratio https://en.wikipedia.org/wiki/Blackstone%27s_ratio In a future with brain-computer interfaces, would you like such an algorithm to search your mind for illegal information too? Is it still your device if it acts against you?