7 ms·
Given all the zero day exploits on iOS I wonder if it's now going to be viable to hack someone's phone and upload child porn to their account. Apple with happil
by marcinzm 5y ago
Given all the zero day exploits on iOS I wonder if it's now going to be viable to hack someone's phone and upload child porn to their account. Apple with happily flag the photos and then, likely, get those people arrested. Now they have to, in practice, prove they were hacked which might be impossible. Will either ruin their reputation or put them in jail for a long time. Given past witch hunts it could be decades before people get exonerated.
- toxik 5y agoThis is really a difficult problem to solve I think. However, I think most people who are prosecuted for CP distribution are hoarding it by the terabyte. It’s hard to claim that you were unaware of that. A couple of gigabytes though? Plausible. And that’s what this CSAM scanner thing is going to find on phones.
- emodendroket 5y agoA couple gigabytes is a lot of photos... and they'd all be showing up in your camera roll. Maybe possible but stretching the bounds of plausibility.
- danachow 5y agoA couple gigabytes is enough to ruin someone’s day but not a lot to surreptitiously transfer, it’s literally seconds. Just backdate them and they may very well go unnoticed.
- tornato7 5y agoIt's also possible to 'hide' photos from the reel in the photos app. Many people are unaware of that feature so an attacker could hide as many photos they want in your iCloud.
- imwillofficial 5y agoAssmuning they have access
- pixl97 5y agoWith the number of security flaws that exist these days we should assume someone always has access that is not intended.
- l33t2328 5y agoHow do you do that?
- runlevel1 5y agoGigs of software updates and podcast episodes are regularly downloaded to phones without being noticed. How frequently do most people look at their camera roll? I'd be surprised if it's more than a few times a week on average. Does an attacker even need access to the phone? If iCloud is syncing your photos, your phone will eventually see all your pictures. Unless I've misunderstood how this works, the attacker only needs access to your iCloud account.
- kelnos 5y ago> I'd be surprised if it's more than a few times a week on average. For me it's probably 5-7 times per day, but I also take a lot of photos. I think a few times a week is probably low-balling it, even for an average.
- Mirioron 5y agoI see my camera reel about once every few months. If I'm not taking a picture I don't see the reel.
- giantrobot 5y agoThe camera roll's defaults display images chronologically based on the image's timestamp. I've got thousands of photos on my phone going back years. If you hack my phone and plant some photos with a sufficiently old timestamp I'd never notice them. I can't imagine my situation is all that uncommon either.
- MinusGix 5y agoAs others have said, people have a lot of photos. It wouldn't be too hard to hide them a bit from obvious view. As well, I rarely look at my gallery unless I need to. I just add a few photos occasionally. So maybe once every two weeks I look at my gallery, plenty of time to initiate that.
- 0x426577617265 5y agoWhy would they hoard it in the camera/iPhotos app? I assume that storage is mostly pictures taken with the device. Wouldn't this be the least likely place to find a hoard of known images?
- new_realist 5y agoThis is already possible using other services (Google Drive, gmail, Instagram, etc.) that already scan for CP.
- t0mas88 5y agoDoes Google scan all files you upload to them with an algorithm like the one now proposed? Or do they have only a list of exact (not perceptual) SHA hashes of files to flag on? The latter I think is also used for pirated movies etc being removed under DMCA?
- bccdee 5y agoI'm pretty sure they use perceptual hashes for matching CSAM. A lot of cloud services do this sort of thing.
- acdha 5y agoYes: it’s called PhotoDNA and is used by many, many services. See https://en.wikipedia.org/wiki/PhotoDNA https://en.wikipedia.org/wiki/PhotoDNA SHA hashes aren’t suitable for this: you can change a single bit in the header to bypass a hash check. Perceptual hashes are designed to survive cropping, rotation, scaling, and embedding but all of those things mean that false-positives become a concern. The real risk would be if someone figured out how to many plausibly innocent collisions where you could send someone a picture which wasn’t obviously contraband or highly suspicious and attempt to convince them to save it.
- seph-reed 5y agoSomeone is going to figure out how to make false positives, and then an entire genre of meme will be born from putting regular memes through a false positive machine, just for the lulz. Someone else could find a way to make every single possible mutation of false positive Goatse/Lemonparty/TubGirl/etc. Then some poor Apple employee has to check those out.
- mirker 5y agoIf Apple is indeed using CNNs, then I don’t see why any of the black-box adversarial attacks used today in ML wouldn’t work. It seems way easier than attacking file hashes, since there are many images in the image space that are viable (e.g., sending a photo of random noise to troll with such an attack seems passable).
- brokenmachine 5y agoWhat's a CNN?
- mirker 5y agoConvolutional Neural Network e.g., deep learning
- 0x426577617265 5y agoIf the process of identifying the images is done on the device, then a jailbroken device will likely give an attacker access to the entire DB. I'm not sure how useful it would be, but if the attacker did have access to actual known CSAM images it probably wouldn't be hard for them to produce false positives and test it against the DB on the jailbroken device, without notifying the company.
- gnopgnip 5y agoWouldn't this risk exist already, as long as it is uploaded to icloud?
- remram 5y agoThe "hack" might be very simple, since I'm sure it's possible to craft images that look like harmless memes but trigger the detection for CP.
- 0x426577617265 5y agoCouldn't the hack just be as simple as sending someone an iMessage with the images attached? Or somehow identify/modify non-illegal images to match the perceptual hash -- since it's not a cryptographic hash.
- barsonme 5y agoDoes iCloud automatically upload iMessage attachments?
- tjoff 5y agoDoesn't need to, the detection is client side at first.
- marcellus23 5y agoNo, like many others commenting on the issue, you seem to only have a vague idea of how it works. Only photos being uploaded to iCloud are being scanned for CSAM.
- tjoff 5y agoThen tell us. Because this is what apple says: The Messages app will use on-device machine learning to warn about sensitive content, while keeping private communications unreadable by Apple. Next, iOS and iPadOS will use new applications of cryptography to help limit the spread of CSAM online, while designing for user privacy. https://www.apple.com/child-safety/ https://www.apple.com/child-safety/ There is no ambiguity here. Of course they will scan images in the cloud as well, but they are explicit in saying that it is (also) on the device itself.
- dylan604 5y ago>Given past witch hunts it could be decades before people get exonerated. Given how pedophiles are treated in prison, that might be longer than your expected lifespan if you are sent to prison because of this. Of course I'm taking it to the dark place, but you kinda gotta, you know?
- TeeMassive 5y agoYou don't even need hacking for this to be abused by malevolent actors. A wife in a bad marriage could simply take nude pictures of their child to falsely accuse her husband. This tech is just ripe for all kind of abuses.
- amannm 5y agoThat picture wouldn't already be in the CSAM database...
- TeeMassive 5y agoThere's no point for this tech then. There's no way that they will no try to expand to detect the presence of porn and children being both present at the same time.