8 ms·
Tell HN: Dont upload these images to iCloud as Apple will assume its Child Porn
- victor871129 5y agoThese harmless generated images have a neuralhash equivalent to those provided in the NCMEC database submitted for testing. I repeat: Dont upload these harmless images to iCloud as Apple will assume its Child Porn (CSAM). Scripts were available on a GitHub repo but were removed because they may cause damage to others.
- ryanlol 5y ago> Scripts were available on a GitHub repo but were removed because they may cause damage to others. Is there an archived link? Edit: I guess this? https://gist.github.com/unrealwill/c480371c3a4bf3abb29856c29197c0be https://gist.github.com/unrealwill/c480371c3a4bf3abb29856c29...
- yreg 5y agoIs the hash database and the hashing algo public? Or how do you know these match?
- zepto 5y ago> Dont upload these harmless images to iCloud as Apple will assume its Child Porn (CSAM) This is not true. They may match the hash, but the will not match the visual derivative. The system is not as easily fooled as you think.
- sharken 5y ago> The system is not as easily fooled as you think. I would like to believe that is true, but the negative consequences of even generating a false-positive is enough to not attempt to upload any image.
- deleted 5y ago[deleted]
- LucidLynx 5y agoI tend to disagree here... Based on the documentation from Apple, they are waiting to get *several* matches, *not only one* (we don't know what is *several* but I don't expect something like <= 3 pictures). Once the rate has been reached, they ask to a physical team to review the "positive matches", and deliberate if, yes or no, the images are CSAM or not. If yes, after the manual process, the authorities are called.
- threatofrain 5y agoHypothetically, what happens if a viral event should persuade people to mass upload these images? Would Apple ad hoc modify their review protocol?
- jareklupinski 5y agoyou mean like how people got so fed up with ToS-mandated arbitration that they all decided to file motions simultaneously it worked that time...
- zepto 5y agoNothing because these files won’t trigger a match.
- krrrh 5y agoIf Google Drive scans with the same database then how is your link working?
- fulldecent2 5y agoBecause they are scanning with a different hash system.
- fresswolf 5y agoInstead, send these images to someone you hate over WhatsApp. Chances are that he/she has automatic image saving activated (default setting). New way of SWATting, and completely legal?
- rvz 5y agoHere's a small tip: You can edit the title to have 'Tell HN' which will more likely bring this post to their attention. Thanks for the heads up.
- jandorn 5y agoAs far as I know they all do CSAM, Google, Microsoft, Facebook, and now Apple with iOS 15. So isn't it already a problem that you have it in a Google Drive?
- victor871129 5y agoI put it in Google Drive on purpose to analyze if Google is also scanning as aggressively as Apple. So far no warnings.
- jbverschoor 5y agoMaybe someone needs to send these to some of the executives to make a point.
- csomar 5y agoDid you try it with Apple, though? Or did anyone else?
- auslegung 5y agoBut what if millions of people do it? Sounds like Little Brother.
- dejw 5y agothen millions of people can f** * because it's apple! didn't google recently close a personal account of an indie game developer without giving him any explanation?
- deleted 5y ago[deleted]
- ugjka 5y agoNot clicking on that link, don't want my G account go poof
- jsnell 5y agoYeah, seems like anyone clicking through is playing with fire. If the description is correct, posting the link is highly irresponsible. (It seems like the right thing to do would be to serve the content from a server the OP controls themselves.)
- YokoSix 5y agoI clicked the link. Those images look like modern, colorful and expressionistic art.
- ugjka 5y agoWith pornographic features... i installed tor-browser
- podric 5y agoMaybe I'm an idiot but my curiosity got the best of me and I clicked the link. The photos just look like abstract modern art, although their perceptual hash may match that of known CSAM, I doubt that anyone who clicked the link will get into legal trouble even if Google flags that folder by detecting a perceptual hash match, as they will likely use real people to verify before taking legal action. Google has been recently focused on cultivating the image that they care about user privacy. The last thing they want to do is call the cops on a bunch of HN users for looking at some abstract swirly pics.
- yreg 5y agoI think the concern here isn’t legal trouble but getting banned by some automated system. I’ve heard from googlers that once an account is nuked for suspected child abuse no one will ever want to touch it to find out whether the ban was legitimate.
- ReactiveJelly 5y ago"Now you can block people in Drive. To prevent people from sharing unwanted files with you, ..." hahaha. What a coincidence, Google! So you got a hold of the neural hashes, and then used an error function and descent to generate images that match a 'hash'? It feels wrong to call them 'hashes' when they're so weak to pre-image attacks. They're not the same idea as cryptographic hashes at all. Also want to underline how spooky it is that some of them do resemble human forms.
- 411111111111111 5y agoSome? Literally all are clearly based on pornography. First is veg&butthole, then boobs, next is doggy style etc etc (edit: it seems the order isn't consistent. So I'm likely seeing different images then you.) You can go through them all and see the original pornography if you look at the shapes. To me, it looks more like they started with the real images and tweaked them to make them artsy.
- yesbabyyes 5y agoI don't know, this sounds a bit like a Rorschach test to me.
- deleted 5y ago[deleted]
- ASalazarMX 5y agoI love this comment. I have no idea how these were generated, but even starting with random noise it's possible to end up with vaguely human shapes if that's what originated the hashes to begin with. These images could be a joke, as I don't think we have a clear technical documentation of how these hashes are generated. Computer vision? Vectors? Face recognition software? It's definitely not a naive hash. Edit: seeing the other comments in this thread referencing Twitter, it looks like it's more naive than expected, as the hash is resistant to resizing, but not to cropping. The implementation can change at Apple's discretion, though.
- 5y ago
- LucidLynx 5y agoDo you have any proof? The database of 200_000 images used by Apple (and others?) is private, and I did not found any trace of the hashes (but I could made a mistake here). So, how do you know that those correspond exactly (or with a certain threshold that has NOT been disclaimed by Apple) to the CSAM DB? Also, NeuralHash has NOT been released by Apple yet (https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...), so...
- victor871129 5y agoIm going to release 5 pieces of proof in the next 5 days
- Retr0spectrum 5y agoThis is a lame attempt at trolling.
- zepto 5y agoThis comment discredits you.
- LucidLynx 5y agoThank you Mr Scam
- Retr0id 5y agoThe NeuralHash code is apparently included in the latest beta: https://twitter.com/KhaosT/status/1424205967122571268 https://twitter.com/KhaosT/status/1424205967122571268
- i_am_proteus 5y agoI've never used modern apple products, but I have a question on how apple cloud works: is it possible that simply sending these in a messenger software to someone who uses apple cloud for automatic backups could get that person flagged as a child pornographist?
- hdjjhhvvhga 5y agoMy understanding is that this is the whole point and one of the main reasons people get so upset about it.
- zepto 5y agoNo, because they won’t actually match. The poster is just wrong.
- robertoandred 5y agoNo, messages and regular file backups aren’t checked for child porn and the online photo library has multiple checks and reviews to prevent any false flagging.
- harikb 5y ago“reviews” - that is another thing public has been crying about by the way
- Spooky23 5y agoThe thing that everyone has their panties in a bunch about here and a that like an antivirus scanner, there will be a hash match to child abuse images when you send it. The current practice is that Apple, Google, Microsoft, etc scan the content of your cloud storage. The scenario that you described is a risk and has been since cloud providers started scanning 10-15 years ago. Some large companies scan their file servers as well.
- zepto 5y agoYes, except that in Apple’s implementation there also a ‘visual derivative’, which is essentially a blurred thumbnail. Both must match to cause a positive. These images may match the neuralshash, although we have no proof of that at all. They will not also match the visual derivative. This whole post is based on incomplete information.
- JacobiX 5y agoThe problem with Apple’s approach to CSAM is that they use Neuralhash. Unlike other simple perceptual hashes, the failure modes and the collisions using this method are not well understood. I repeat here my previous comment in another thread : they use NN and triplet embedding loss, the exact same techniques used by neural networks for face recognition, so maybe the same shortcomings would apply here. For example a team of researchers found a 'Master Faces' that can bypass over 40% of Facial ID. Now suppose that you have such an image in your photo library, it would generate a ton of false positives and not just a single match with the NCMEC database.
- deleted 5y ago[deleted]
- hdjjhhvvhga 5y agoIs this true? I have no idea how to even test it without causing problems to myself. After all, they pushed me to give them my credit card and physical address.
- paulcole 5y ago“Your unlaminated, out-of-state driver's license is proof enough for me.” There really is a Simpson’s quote for everything.
- zepto 5y agoAs far as I can see the claim made here is not correct. Assuming the images do as claimed match the hash, they must also match the ‘visual derivative’ in order to trigger a match. The system isn’t as easily fooled as is being claimed here.
- toxik 5y agoYou have misunderstood. NeuralHash is the visual derivative. Read [1] carefully, it's a very confusing document even for experts - nowhere is there a second step to this process where some second type of "visual derivative" is matched. The NeuralHash is what matters, solely. [1] https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
- malf 5y agoThere’s literally a page showing “NeuralHash + visual derivative” in the thing you posted.
- toxik 5y agoEdit: I did find it, but only because somebody else pointed it out. Guess I suck at reading. Also search for "derivative" failed to find it. This thread was good https://twitter.com/fayfiftynine/status/1427900272148246530 https://twitter.com/fayfiftynine/status/1427900272148246530 Be specific, because I cannot find it.
- YokoSix 5y agoSomeone please post this to the Apple subreddit. I'd love to see hell breaking loose over there.
- yreg 5y agoWhat do you mean by "hell breaking loose"? /r/Apple talks about this topic a lot and, similar to HN, is not happy about it. This drive link brings very little additional light to what was already known and discussed.
- post_break 5y agoThe sub is in crisis mode right now. Normally a huge pro-apple even on their worst days do no wrong sub. Right now people are pissed and speaking up. The mods are pissed that they have to deal with it (boo hoo). Anything to add fuel to this fire is good in my eyes because the same people who said "It's ok I have nothing to hide" from Edward Snowdens work are getting slapped in the face for their same ideology to this happening to Apple.
- yreg 5y ago>"It's ok I have nothing to hide" I don't remember that ever being a popular take on reddit. But still, how do you know it's "the same people"? There are a lot of users who hold all kind of opinions.
- post_break 5y agoI guess I meant that figuratively. The same genre of people who thought prism is ok because they have nothing to hide to give up privacy are now the same genre that think this privacy invasion is ok as well.
- yreg 5y agoAh I see, yes that makes sense.
- mcintyre1994 5y agoWhat hash algorithm are they using? If this is legitimate (I’ll be honest - I’m not clicking) then surely any hash this easy to pre-image attack is completely useless? Why wouldn’t they be using a cryptographic hash here?
- soneil 5y agoAs I understand it the NN "perceptual hash" is supposed to hash the image, not the file. eg, if I take a photo of my cat, and hash the file. Then remove my geo data from the exif - the hash no longer matches. It is still very clearly my cat, but cryptographic hashes don't match. This could be resizing the image, saving as png, mirroring/flipping it, etc. The "perceptual hash" should be able to say "no, that's still the same image" while the file data has been entirely transformed.
- float4 5y agoJust uploaded them all to my iCloud. They way I see it, this is the best photo backup approach one can possibly take. Just get flagged for child porn, and have all your iPhone photos stored indefinitely on FBI servers. Does the FBI have geo-redundancy?
- YokoSix 5y agoThanks for your service.
- zoba 5y agoHow do you generate an image like this from a hash?
- newscracker 5y agoIs this obtained from a set of images leaked from NCMEC and regenerated to match the expectations of the as yet (and probably forever) unknown Apple’s NeuralHash algorithm as well as the threshold used to flag content for internal human review for a system that’s going to be operational for U.S. Apple device users only when iOS 15 is released? On what basis is a set of forest-like and post-alien-invasion and post-apocalyptic abstract art is going to get flagged (my poor eyes see one or two that could have some symbolism)?
- emerged 5y agoI think someone needs to play big techs own game and find some cases where this algorithm underperforms based on race or gender, publish a bunch of clickbait articles and get the whole program canceled. Erosion of privacy and authoritarianism isn’t enough to gain traction.
- deleted 5y ago[deleted]
- northisup 5y agoSo will everyone. The whole industry uses the same algorithm to calculate hashes.
- northisup 5y agoSo will everyone. The whole industry uses the same algorithm to calculate hashes. Then the list gets more accurate and we move on.
- ithinkimgood 5y agoThis is so fake it's not even funny. These are just images generated by the model from https://thisartworkdoesnotexist.com https://thisartworkdoesnotexist.com . It's hilarious to see so many people falling for it here
- fulldecent2 5y agoMissing a key feature ~~ there should be 30+ images. You need to have that many to flag an account.
- cryptonector 5y agoWhy was this flagged? I'm sure there are good reasons, I just want to know. EDIT: Oh, I mixed up tabs. This is a link to a google drive of pictures. Because I have scripts disabled, I got no thumbnails, and I'm thinking since this was flagged, maybe I really don't want to get any thumbnails.
- Retr0spectrum 5y agoThis post is still getting linked from other places, so I think it's helpful to point out that it's almost certainly fake. (Hence its [flagged] status) The images shown do appear to be adversarially generated inputs against some NN-based image hash or classifier, but there is no evidence to suggest that this is at all related to Apple's NeuralHash, or that the colliding hashes are from a real CSAM database (the target hashes are not public). OP claimed they would "release 5 pieces of proof in the next 5 days" [1], and guess what, 11 days later they still haven't. Look at OP's post and comment history, it's quite clear that they are a troll. In the mean time, it has been actually proven that hash collisions against NeuralHash are trivially possible, see [2] [1] https://news.ycombinator.com/item?id=28107393 https://news.ycombinator.com/item?id=28107393 [2] https://github.com/anishathalye/neural-hash-collider https://github.com/anishathalye/neural-hash-collider