7 ms·
Why are exact collisions interesting? They are not intended to be compared exactly. This algorithm doesn't even give exact matches for the same image on differ
by scotty79 5y ago
Why are exact collisions interesting? They are not intended to be compared exactly.
This algorithm doesn't even give exact matches for the same image on different hardware.
https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX
Note: Neural hash generated here might be a few bits off from one generated on an iOS device. This is expected since different iOS devices generate slightly different hashes anyway. The reason is that neural networks are based on floating-point calculations. The accuracy is highly dependent on the hardware. For smaller networks it won't make any difference. But NeuralHash has 200+ layers, resulting in significant cumulative errors.
- yeldarb 5y agoThat is a good point; has Apple stated how many bits two images’ NeuralHashes can differ by and still be considered a “match” by their system?
- tgv 5y agoThe hash is 96 bits long. When hashing 1 billion pictures, that gives a collision probability of 6e-12. If it were uniformly distributed. There's no way people have hashed billions of images already. It just shows that it's pretty probably there will be collisions, and on visual inspection, it looks as if the collisions will happen on visually similar images. So if there's a naked baby pic in the CSAM database, quite a few of you 100s of child pictures can be flagged.
- Lamad123 5y agoThe Tim cuk's law: "If it can happen, it will happen"
- halflings 5y agoClearly this is not a cryptographic hash, and hence it's known hashes are not uniformly distributed. Apple explained in their technical summary [0] that they'll only consider this an offence if a certain number of hashes match. They estimated the likelihood of false positives there (they don't explain which dataset was used, but it was non-CSAM naturally) is 1 out of a trillion [1] In the very unlikely event where that 1 in a trillion occurrence happens, they have manual operators to check each of these photos. They also have a private model (unavailable to the public) to double-check these perceptual hashes which also used before alerting authorities. [0] https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni... [1] https://www.zdnet.com/article/apple-to-tune-csam-system-to-keep-one-in-a-trillion-false-positive-deactivation-threshold/ https://www.zdnet.com/article/apple-to-tune-csam-system-to-k...
- tgv 5y ago> they have manual operators to check each of these photos For now. But what will happen when there are thousands of false positives per day? Will they increase the staff? Or will they add another algorithmic layer? Or just up the threshold a bit? There's no guarantee. The only thing that's certain is that the NeuralHash doesn't inspire confidence.
- snowwrestler 5y agoRegardless of what Apple does, law enforcement must always manually review suspected CSAM before requesting a warrant based on it. So the idea that you could SWAT someone with some hash collisions on innocent images is just not possible. At most you could maybe temporarily lock someone’s iCloud account. But again, the collisions would need to be multiple and all look like CSAM at reduced resolution. In general, it seems not correct to think about NeuralHash like SHA or RSA. It’s not a cryptographic system and collisions are not a one-step endgame.
- bigwavedave 5y ago> Regardless of what Apple does, law enforcement must always manually review suspected CSAM before requesting a warrant based on it. So the idea that you could SWAT someone with some hash collisions on innocent images is just not possible. A quick search found four clear cases where law enforcement has favored technological false positives over evidence: Ousmane Bah: https://www.businessinsider.com/teen-sues-apple-1-billion-false-arrest-facial-recognition-2019-4?op=1 https://www.businessinsider.com/teen-sues-apple-1-billion-fa... Robert Williams: https://www.cbsnews.com/news/facial-recognition-60-minutes-2021-05-16/#app https://www.cbsnews.com/news/facial-recognition-60-minutes-2... Nijeer Parks: https://www.cnn.com/2021/04/29/tech/nijeer-parks-facial-recognition-police-arrest/index.html https://www.cnn.com/2021/04/29/tech/nijeer-parks-facial-reco... Michael Oliver: https://www.dailydot.com/debug/detroit-facial-recognition-wrongful-arrest/ https://www.dailydot.com/debug/detroit-facial-recognition-wr... The one on Ousmane Bah really frustrates me- not only was he on a date at prom during the theft, he was in another state! "Nothing to hide" does not mean "nothing to fear" and allegations (even false ones) of possessing CSAM will ruin lives. At the end of the day, what it really comes down to is trust; personally, I do not have enough faith in due process to not ruin innocent people. But I'm just some guy online, I will readily admit I don't know anything about anything.
- ajb 5y agoIf you can get exact collisions, this can be gamed. For example, suppose there are two rival gangsters. One wants to set the police on his rival. He knows that a certain (innocuous) image is on his rival's phone. So he pays someone to generate a fake child-porn image with the same neuralhash, and ensure that it gets into the child porn DB. Then, apple reports the rival to the police, and they come and investigate him. OF course, they may notice that the image isn't the right one, but by that time they may have found other incriminating evidence. Not sympathetic to a rival gangster? Ok lets find an innocent victim: not a rival criminal, but an innocent witness who our protag wants to intimidate. Gangster wants to intimidate the witness, but can't get at them, so cooks up a scheme to convince the witness that the police are in his pocket. Exactly as above, causing the police to investigate the witnesses phone. Another one might be, a certain government wants to identify opposition groups using images associated with them . Apple is not keen to be associated with that, but the government can simply generate fake child-porn (remember, programmatically generated CP is just as illegal) for each image of interest.
- monkeynotes 5y agoI think before a criminal investigation, or any investigation at all is pursued, a human verifying the images would dismiss the false positive. I would think surreptitiously placing actual child porn on a rival's phone/computer would be much, much more effective. Cybercriminals could likely do all this remotely. Phish for apple account login, upload images. Done.
- iamtheworstdev 5y agoyou think the US government would respect the law before initiation of actions to take down a criminal? they rarely do that when dealing with people that dont have criminal backgrounds
- zepto 5y agoIf you really believe you live in a country with a completely corrupt legal system, then this stuff is the least of your worries.
- specialist 5y agoPer ATP: Apple will compare hashes of local photos with a national registry of child pornography photos. Once a certain (unknown) threshold is reached, let's say 20 hits, some kind of escalation occurs, with some kind of manual (human) review steps. Accidental Tech Podcast - A Storm of Asterisks https://atp.fm/443 https://atp.fm/443 I haven't listened to the follow up episode yet. I still have zero opinion on this photo scanning kerfuffle. I just don't know enough. Of all the "hot takes" on this issue, ATP's has been the most comprehensive. So appreciated.
- pbronez 5y agoThanks for the link. Daring Fireball’s analysis is the most helpful take I’ve seen on the topic so far: https://daringfireball.net/2021/08/apple_child_safety_initiatives_slippery_slope https://daringfireball.net/2021/08/apple_child_safety_initia...
- hackinthebochs 5y agoI'm interested in the details of the manual review. Does Apple have access to the database of original images and will they use it compare? If not, I can imagine a scenario where a photo of a naked child is flagged as matching the database, the human reviewer sees that it is in fact a naked child and assumes this must mean the image has been legally determined to be child pornography. The case gets referred to authorities and the innocent victim's life is upended for potentially years until the case plays out.
- snowwrestler 5y agoLaw enforcement must manually review all suspected CSAM before seeking a warrant based on it. There is a whole process there, beyond whatever Apple has implemented, before a prosecution begins. Understand that matching a file in the NCMEC database is not itself a crime. The whole CSAM-detecting ecosystem is just a tool for surfacing potential crimes. Having a few pics of your own child naked is not illegal and it’s pretty easy for law enforcement to figure out if that’s the case.
- 5y ago
- cannabis_sam 5y ago> Why are exact collisions interesting? 1. What does “exact” mean to you in this context? 2. What else is more interesting about a hashing algorithm used to identify things, other than its collision rate?
- scotty79 5y agoAd.1 Same value of each of 96 bits of the hash. Ad.2 If hashes are to be matched approximately, not exactly, for example will be considered a match if they differ in less than 3 bits out of 96 then the most interesting thing should be how many collisions you can find if you compare them like that.
- chefandy 5y ago... does it have to prove Apple wrong about something to be interesting?
- GistNoesis 5y agoAfiu, Apple's NeuralHash uses exact collisions when they do their Private Set Intersection. The main advantage of using exact collision is that you can then blind the perceptual hash with a cryptographic hash and avoid any leak of information. (Taking for example sha256 of this perceptual hash won't allow any attacker to get any information on the features from the hash, but if the perceptual hash are the same then the input of the sha256 is the same and therefore the output of the sha256 is the same). This is important because it alleviates the risks of an eventual leaking the database as Apple never touched and compared sensitive content but only cryptographic hashes of the perceptual hashes. Some other system like PhotoDNA, rely on a euclidean distance between features being less than a threshold to register a match, which allows to quantify how far the image is from CSAM, but mean that the hash leak some information about the original content.
- scotty79 5y agoBut they can't use exact matches because their algorithm doesn't create same hash for same image on their various platforms due to hardware differences in floating point arithmetic on their different devices. Unless they have on their servers floating point arithmetic emulators that can calculate exact hash for each of their different devices for each offending image then they can't only match exactly.
- umanwizard 5y agoThat explanation doesn’t make sense to me. Standard (IEEE) floating-point math is deterministic; it shouldn’t depend on the hardware.
- user-the-name 5y agoThe processors that implement hardware-accelerated neural nets are not necessarily using standard IEEE floating-point math.
- nullc 5y ago> They are not intended to be compared exactly. Apple's private set intersection which leaks the keys to decrypt the images coniditional a neuralhash match requires an exact match. They probably didn't realize they got different results on different toolchains/devices, since they target a mono-culture and the whole subsystem shows fairly little careful thought went into it. They could easily make an exact integerized version which would be consistent. It would still be broken. :)