4 ms·
The integrity of this entire system now relies on the security of the CSAM hash database, which has just dramatically increased in value to potential attackers.
by vermilingua 5y ago
The integrity of this entire system now relies on the security of the CSAM hash database, which has just dramatically increased in value to potential attackers.
All it would take now, is for one CSAM hash to be known to the public, then uploading collided iPhone wallpapers to wallpaper download sites. That many false positives will overload whatever administrative capacity there is to review reports in a matter of days.
- Matheus28 5y agoThere's no need for someone to get the entire CSAM database. If they go on the darknet and just find enough images (or hashes) that would trip Apple's system, that would be enough. I'd assume any publicly available image on the darknet would likely also be on CSAM.
- copperx 5y agoExactly. It would be trivial for anyone to compile a list of possible known CSAM hashes. It would be illegal to do so, but only one person has to do it, and then the list of probably positive hashes can be distributed legally around the web.
- gnulinux 5y agoHuman doesn't need to see the material. One can automate this by crawling websites with known bad material and compute hash of all images. Surely some of those hashes are CSAM, we don't know which. But it's still equally dangerous.
- simondotau 5y agoBefore they make it to human review, photos in decrypted vouchers have to pass the CSAM match against a second classifier that Apple keeps to itself.
- ec109685 5y agoNo, there’s another private hash function that also has to match the known CSAM image for an image to be considered a match. That one can’t be figured out through this technique.
- Syonyk 5y agoThey couldn't possibly be using the "other perceptual hash algorithm commonly used for this stuff," PhotoDNA, could they? I mean, hopefully not, but at this point, it's reasonable to call just about everything into question on the topic.
- ec109685 5y agoTheir goal is to minimize false positives, so would not be in their best interest.
- vermilingua 5y agoBut surely if they're doing this in anticipation of E2EE user data, that procedure becomes moot? So either they have no intention to actually protect user data, or the system is trivially broken; either way a pretty damning look for Apple.
- comex 5y agoNo, because the second hash is only performed after the first hash has already matched, and so, using the threshold secret sharing crypto, the server has learned the escrowed encryption key for the image. (Well, for its “visual derivative”, at any rate.)
- vermilingua 5y agoSo, Apple retains the ability to decrypt E2EE data? That’s… worse?
- ec109685 5y ago
- Fnoord 5y agoTo assume CP is reviewed manually is simply wrong. You don't want to put such weight on an individual. You want to automate it as much as possible, with as little false positives (and false negatives) as possible. For example, in case of a wallpaper, let's say its the Windows XP wallpaper. There's no human skin color in it at all, so you can easily be reasonably sure it isn't CP. You would not need an advanced ML for such. And they can have multiple checksums, just like a tarball or package or whatever can have an CRC32, MD5, and SHA512. Just because one of these matches, doesn't mean the other don't. Only problem is keeping these DBs of hashes secret. But that could very well be a reason the scanning isn't done locally.
- vermilingua 5y agoTo assume it’s never reviewed manually is absolutely terrifying.
- Fnoord 5y agoThat is not what I asserted. I asserted that it is only done when other options are exhausted.
- user-the-name 5y agoApple explicitly states that it is reviewed manually.