3 ms·
It took me time to think about this news. I despise the cavalier attitude some people in software engineering have towards the social consequences of their work
by yyyk 5y ago
It took me time to think about this news. I despise the cavalier attitude some people in software engineering have towards the social consequences of their work. A community where 'think of the children' is an ironic phrase is a sick one. Yet, something here doesn't smell entirely right.
TL;DR: I think one of three must be correct:
A) NCMEC is seriously neglecting protecting children by accepting a subpar solution. We're getting the client-side scanning precedent without any benefit to society.
OR
B) Apple is not entirely ditching server-side scanning, so again we'll be gaining nothing from client-side scanning aside from normalization of a dubious practice.
OR
C) The client-side solution will end up way more invasive than what even Apple believes it would be.
---
Lets look at the worst scenarios. One or more pedos is actively molesting in order to record CSAM on a iPhone. The iPhone soon syncs to the iCloud, while the pedos distribute the CSAM. This is the scenario we all want to stop most and ASAP.
1) Apple's is a hash-based approach, not an AI based one. So the original upload must be marked as 'clear' by the iPhone, since NCMEC can't yet know about the CSAM.
2) Hopefully, sometimes later NCMEC does find the CSAM and marks it appropriately.
3) In the 'naive' server-side approach, Apple could scan iCloud and find the original CSAM and then the uploaders. This is not what we're doing. This is a client-side approach, and if certain rumors are right Apple will eventually do end2end encryption and so will not even be able to scan server-side. So do we do?
4) One option is for NCMEC to give up and let Apple do a worse solution than server-side scanning where the original perpetrators not only go free but can use iCloud for storage without being detected. That's our option A above.
This is undesirable and unlikely. Even if NCMEC were so compromised, Apple can't want to get all this fire and then implement an non-working solution. Even if Apple were to do that, eventually pressure will force them to implement a working solution.
5) The other option is for the iPhone client to receive regular hash updates, and then periodically rescan any picture that was at one time uploaded to the iCloud (option C). Since the incriminating files may well not even exist anymore on the phone (the pedos in our scenario have good motivation to eventually delete the CSAM to save space even without the issue of client-side scanning), Apple will have to store the hashes of deleted images on the phone in order to periodically rescan them.
So to implement effective client-side scanning, iPhone owners will not be able to ever truly delete an image if it was ever synced to iCloud, even if the owners deleted it from the cloud and their phone. The hash will be stored, and not only that - a rough facsimile of the image should be restorable from the hash*. I can't see any other way to do it, and Apple and NCMEC must want the client-side solution to work. In a world where Pegasus and things like it exist, this is.. not optimal.
6) Of course, Apple could avoid all this by storing the hashes in their iCloud (option B), but what would be the point of client-side scanning than, aside from a dubious precedent?
Am I missing something?
* AFAIK the ability to make a rough facsimile of the original image from the hash is a property of visual hashes required for them to work even on manipulated images, and also required for the manual review system. Apple can't avoid this on a client-side solution - it's either the original image or something close enough so it can be manually reviewed if later found to be CSAM.
- yyyk 5y agoThinking about it, another client side solution would be to also scan images when downloading from iCloud. That way pedos will have difficulty using iCloud, since the hash will be rescanned when downloading and the hash might be up to date this time. Would this allow Apple to not store the deleted image data on the client? This would still catch significantly less perpetrators than the server-side approach, would NCMEC etc. agree to this, or would pressure force Apple to go even further? Either way, a workable client-side solution has to end up more encompassing than what Apple has done.