3 ms·

"Apple said that there is a one in one trillion chance of a false positive" https://techcrunch.com/2021/08/05/apple-icloud-photos-scanning/ https://techcrunch.

"Apple said that there is a one in one trillion chance of a false positive" https://techcrunch.com/2021/08/05/apple-icloud-photos-scanning/ https://techcrunch.com/2021/08/05/apple-icloud-photos-scanni... Versus: https://blog.roboflow.com/nerualhash-collision/ https://blog.roboflow.com/nerualhash-collision/ Along with: https://github.com/anishathalye/neural-hash-collider https://github.com/anishathalye/neural-hash-collider Demonstrates blatant incompetence on the part of Apple's design. At this point, trust is broken. That's the end of credibility regarding any other claim made about the security or base function of the system. As for a simple example of one of many potential attack vectors (search HN for "neuralhash" and read the comments for countless examples), here is a basic attack: As Apple's hash algorithm is entirely broken and useless, a malicious actor can easily craft a NSFW, but otherwise legal image, that has a hash matching one in the CSAM database, that is visually ambiguous in that context (for example, a close-up). Targeted attacks could be launched on unsuspecting victims as simply as sending them an image. If their app is configured to auto-save to iCloud - as some are by default, and as many are configurable to be - the image will then get automatically flagged as a match, and may well potentially pass a human review - because the image may indeed look just like CP - and an innocent person, who may not even be aware of the presence of the image in their iCloud library at this point, may get visited by the police. That in itself is bad enough. However it gets worse - do bear in mind that "the police" is not an equivalent concept across nations, yet iPhones are ubiquitously used across the world. In certain countries, a "visit from the police" over such automatically-flagged content could well result in presumption of guilt to the point of immediately administered punishment and a destroyed innocent life. In China for example, conviction rates are routinely 99.99% - being accused of a crime there is equivalent to being convicted, and that's just one major country containing well over 100 million iphone users. Apple is demonstrating not only gross incompetence in this design of this "system", but vagrant disregard for human rights, along with utterly destroying their own long-cultivated pro-privacy stance. It's a disaster for them and their customers in every possible way it could be.

hannasanarion 5y ago
Apple was talking about the chance of accidental false positives. There is a 1 in 4 billion chance that if I send a message to an IP address, it will be yours. The fact that I can know your IP address and send you a message intentionally does not make the previous statement false. > As Apple's hash algorithm is entirely broken and useless, a malicious actor can easily craft a NSFW, but otherwise legal image, that has a hash matching one in the CSAM database, that is visually ambiguous in that context (for example, a close-up). Targeted attacks could be launched on unsuspecting victims as simply as sending them an image. Everybody keeps talking about this scenario as if it is any different whatsoever from sending people actual regular old-fashioned CSAM, which A: already happens, B: requires less effort on the part of the attacker, and C: isn't reliant on the recipient uploading them to iCloud, triggering several layers of filtering and detection, which may or may not result in law enforcement interest depending on your forgery's ability to fool professional investigators at NCMEC who are comparing it alongside the original. The attack you're describing is less effective and more difficult than existing ones.
- argvargc 5y ago
  No, the attacker is committing a crime with the attack you describe, but has plausible deniability that they are not with the Apple-nonsense enabled attack vector. This is a huge difference as not only does Apple facilitate a gain-of-function in terms of potential detriment to the victim of an attack, the newly-enabled vector also indemnifies the attacker. The effort required to come up with a hash collision in SHA256 is something on the order of the power output of a trillion suns for a billion years (or some other meaninglessly-prohibitively-large amount). Despite such metrics and off-the-shelf solutions being readily attainable within this field Apple has implemented a system that has resulted in a collision within days, followed by a free automated collision generation tool. This demonstrates absolute gross incompetence in design, and permanently eliminates any trust any sensible individual can ever have in the system despite your protestations. There is simply nothing you can say to convince any sane person otherwise, the situation has been factually described and it is what it is.

hannasanarion 5y ago

Apple was talking about the chance of accidental false positives. There is a 1 in 4 billion chance that if I send a message to an IP address, it will be yours. The fact that I can know your IP address and send you a message intentionally does not make the previous statement false. > As Apple's hash algorithm is entirely broken and useless, a malicious actor can easily craft a NSFW, but otherwise legal image, that has a hash matching one in the CSAM database, that is visually ambiguous in that context (for example, a close-up). Targeted attacks could be launched on unsuspecting victims as simply as sending them an image. Everybody keeps talking about this scenario as if it is any different whatsoever from sending people actual regular old-fashioned CSAM, which A: already happens, B: requires less effort on the part of the attacker, and C: isn't reliant on the recipient uploading them to iCloud, triggering several layers of filtering and detection, which may or may not result in law enforcement interest depending on your forgery's ability to fool professional investigators at NCMEC who are comparing it alongside the original. The attack you're describing is less effective and more difficult than existing ones.

argvargc 5y ago
No, the attacker is committing a crime with the attack you describe, but has plausible deniability that they are not with the Apple-nonsense enabled attack vector. This is a huge difference as not only does Apple facilitate a gain-of-function in terms of potential detriment to the victim of an attack, the newly-enabled vector also indemnifies the attacker. The effort required to come up with a hash collision in SHA256 is something on the order of the power output of a trillion suns for a billion years (or some other meaninglessly-prohibitively-large amount). Despite such metrics and off-the-shelf solutions being readily attainable within this field Apple has implemented a system that has resulted in a collision within days, followed by a free automated collision generation tool. This demonstrates absolute gross incompetence in design, and permanently eliminates any trust any sensible individual can ever have in the system despite your protestations. There is simply nothing you can say to convince any sane person otherwise, the situation has been factually described and it is what it is.