4 ms·
I'm waiting for orthodox authorities to do stuff like reporting pictures of people (not just girls) without head coverings. or dressed in bikinis, thongs, or un
by ChrisMarshallNY 5y ago
I'm waiting for orthodox authorities to do stuff like reporting pictures of people (not just girls) without head coverings. or dressed in bikinis, thongs, or underwear (like a significant number of "influencers" do, every day).
There are already places in this world, where a couple can be arrested for kissing in public. I suspect that the folks enforcing those laws, would have some real interest in this capability.
Not to mention nations (many in Africa, but there are also Eurasian nations), where homosexuality is banned (or even considered for death penalties). If your phone has a picture of two men (or women) embracing, it could cause nightmares.
- jll29 5y agoOr manipulating photos a little so the classifier deems them dangerous: https://www.youtube.com/watch?v=YXy6oX1iNoA https://www.youtube.com/watch?v=YXy6oX1iNoA
- isaacimagine 5y agoOr sending manipulated but otherwise innocuous photos to someone else so the classifier is triggered on their device. Yikes.
- judge2020 5y agoThis isn’t a neutral network classifier, it’s an algorithm that searches against existing CSAM photos. https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
- ChrisMarshallNY 5y agoThat's potentially, even worse. It means that the 'relevant authorities" can substitute any images they want, whenever they want.
- tomas_nort 5y ago“ Messages uses on-device machine learning to analyze image attachments and determine if a photo is sexually explicit.” https://www.apple.com/child-safety/ https://www.apple.com/child-safety/
- robertoandred 5y agoThat’s not at all how this works. It doesn’t scan for images of arbitrary subjects. It scans for exact matches of known CP. Your vacation pics are in no danger of being flagged.
- jjcon 5y agoScans for exact matches of hashes of photos provided by the government. Apple is not controlling the hash database.
- ChrisMarshallNY 5y agoI don't think it's an exact hash scan. If so, it would be trivial to defeat. People would simply need to do a 1-pixel crop, or recompress.
- robertoandred 5y agoYou’re correct that it’s not a pixel-by-pixel hash, but it’s still a hash of that specific image. It’s not analyzing the image subject and trying to identify it as CSAM.
- randcraw 5y agoIn the real world, exact matching of image hash values won't work. It's routine for images to change in small ways over their lifetimes as they're shared. According to the model you suggest, modifying a single pixel in the image by even the smallest amount would cause a hash mismatch against the original. If Apple's system is truly that inflexible, it will be trivial to circumvent in no time. Just increment / decrement a random RGB pixel in each of your images, and voila, your porn is scot free. Of course this countermeasure will be employed almost instantly by miscreants, so how will the FBI respond? Will they give up? Certainly not. They already have a blank check to spy on our phones. So they will devise a clumsier match algorithm that scans more sources of data on your phone and your cloud accounts and your backups, and produces more false positives. Why wouldn't they do this? Once any telecom service provider opens a door which compromises security or privacy, they will have a much harder time closing it.
- 5y ago