4 ms·

Apple plans to scan US iPhones for child abuse imagery

53 points by jdmark 5y ago

jsnell 5y ago
https://news.ycombinator.com/item?id=28068741 https://news.ycombinator.com/item?id=28068741 (640 comments) https://news.ycombinator.com/item?id=28075021 https://news.ycombinator.com/item?id=28075021 (296 comments)
randomluck040 5y ago
The headlines are marginally misleading. They won’t see actual images but hashes. They probably have a database including hashes of child abuse imagery which they will compare/try to match to the hashes of the images on your phone. If you take an image of your child’s bathing time to send it to your parents so they can see the little bugger during covid, it most probably won’t be a problem. It’s an important distinction in my opinion and it’s happening on a lot of cloud services already. The breach of trust and going for the images on my personal device however is not acceptable for me. As a non-US citizen living outside of the US I’d refuse to use any Apple devices anymore. It’s contradicting their own privacy talk as well which, looking at some parts of the world, is evidently “talk” to some degree. Child abuse is probably the worst crime there is, I agree wholeheartedly, but this will create a back door like situation for every government on earth.
- tomas_nort 5y ago
  Incorrect. “ Messages uses on-device machine learning to analyze image attachments and determine if a photo is sexually explicit.”[0] They do not state this explicitly, but utilizing this technology a mere visit to a website theoretically could result in your home search by the law enforcement of your country. —- [0] - https://www.apple.com/child-safety/ https://www.apple.com/child-safety/
- SN76477 5y ago
  Right, as I understand it, they are comparing images to known abuse, not scanning images looking for "new" abuse.
  - Akronymus 5y ago
    Images retrieved from abusers devices* At least from my understanding. So there are probably a lot of innocent pictures in the DB.
- throwaway0a5e 5y ago
  >They won’t see actual images but hashes Oh great, even better. /s How many billions of images does apple deal with? There's bound to be a heck of a lot of collisions in there.
  - new_realist 5y ago
    This has already been considered and addressed via a secret sharing threshold scheme.
  - voldacar 5y ago
    furthermore they are perceptual hashes. its not like you can just defeat it by changing a pixel in all your images
- taylorius 5y ago
  And how will they see these hashes? By uploading a hash of every file on your iphone to a database in the cloud. Oh - don't worry - they promise to delete them all after checking for child porn. Scouts honour.
  - olliej 5y ago
    It explicitly says the phone downloads the set of hashes and does the computation on the client side. That said I agree that this is going to be hideously abused by every government.
deleted 5y ago
[deleted]
cbanek 5y ago
As much as I hate child abuse (and I really do) I don't like the privacy aspects of this. I feel like this could also be troubling in that people who send you unsolicited images / emails / texts with pictures that might be downloaded to your device could trip this up. I'm already wondering how a hash collision attack could defeat this, although then I can already hear the judge saying "well, you had it there, but you deleted it, so now we'll also charge you with destruction of evidence." This is going to get very interesting very quick as soon as this is first used in case law.
taylorius 5y ago
Who could object to a measure to eliminate child pornography? Unless you're a fan of it, right? And so the precedent is set - and before you know it, they will be checking your phone for all sorts of things.
- matt-attack 5y ago
  Well I personally am perfectly fine with several types of child pornography: 1. Drawn or digitally rendered 2. Photos of me personally as a child (bath time, whatnot) I have no problem with photos of me being used by anyone who is unfortunate enough to be born with an attraction to children. Who on this planet can say that I don’t have the right to distribute my own photos of myself?
  - aaomidi 5y ago
    > Photos of me personally as a child (bath time, whatnot) This is not CP. Although because there is a human factor to this, I wouldn't be surprised if people got harassed over it. Nudist families are well aware of how shitty NGOs and GOs deal with nudity.
simonblack 5y ago
More and more Government interference that harms company sales. Why buy an Apple? Get some other one instead. Nobody will want even the slightest chance of a false-positive ruining their reputation.
ttflee 5y ago
This type of system is just ready-to-use for a totalitarian regime with minor configuration modifications.
cardosof 5y ago
We traded away privacy for "free" apps and services and now we will also give away privacy to help noble causes. Ok, I just hope the loss of privacy does not cause any harm.
xtiansimon 5y ago
What could go wrong? https://news.ycombinator.com/item?id=27696369 https://news.ycombinator.com/item?id=27696369
john579 5y ago
They are scanning phones for porn material they can use for profit, blackmail, and their own perverted entertainment.

53 points by jdmark 5y ago

https://news.ycombinator.com/item?id=28068741 https://news.ycombinator.com/item?id=28068741 (640 comments) https://news.ycombinator.com/item?id=28075021 https://news.ycombinator.com/item?id=28075021 (296 comments)

randomluck040 5y ago

The headlines are marginally misleading. They won’t see actual images but hashes. They probably have a database including hashes of child abuse imagery which they will compare/try to match to the hashes of the images on your phone. If you take an image of your child’s bathing time to send it to your parents so they can see the little bugger during covid, it most probably won’t be a problem. It’s an important distinction in my opinion and it’s happening on a lot of cloud services already. The breach of trust and going for the images on my personal device however is not acceptable for me. As a non-US citizen living outside of the US I’d refuse to use any Apple devices anymore. It’s contradicting their own privacy talk as well which, looking at some parts of the world, is evidently “talk” to some degree. Child abuse is probably the worst crime there is, I agree wholeheartedly, but this will create a back door like situation for every government on earth.

tomas_nort 5y ago
Incorrect. “ Messages uses on-device machine learning to analyze image attachments and determine if a photo is sexually explicit.”[0] They do not state this explicitly, but utilizing this technology a mere visit to a website theoretically could result in your home search by the law enforcement of your country. —- [0] - https://www.apple.com/child-safety/ https://www.apple.com/child-safety/
SN76477 5y ago
Right, as I understand it, they are comparing images to known abuse, not scanning images looking for "new" abuse.
- Akronymus 5y ago
  Images retrieved from abusers devices* At least from my understanding. So there are probably a lot of innocent pictures in the DB.
throwaway0a5e 5y ago
>They won’t see actual images but hashes Oh great, even better. /s How many billions of images does apple deal with? There's bound to be a heck of a lot of collisions in there.
- new_realist 5y ago
  This has already been considered and addressed via a secret sharing threshold scheme.
- voldacar 5y ago
  furthermore they are perceptual hashes. its not like you can just defeat it by changing a pixel in all your images
taylorius 5y ago
And how will they see these hashes? By uploading a hash of every file on your iphone to a database in the cloud. Oh - don't worry - they promise to delete them all after checking for child porn. Scouts honour.
- olliej 5y ago
  It explicitly says the phone downloads the set of hashes and does the computation on the client side. That said I agree that this is going to be hideously abused by every government.

deleted 5y ago

[deleted]

cbanek 5y ago

As much as I hate child abuse (and I really do) I don't like the privacy aspects of this. I feel like this could also be troubling in that people who send you unsolicited images / emails / texts with pictures that might be downloaded to your device could trip this up. I'm already wondering how a hash collision attack could defeat this, although then I can already hear the judge saying "well, you had it there, but you deleted it, so now we'll also charge you with destruction of evidence." This is going to get very interesting very quick as soon as this is first used in case law.

taylorius 5y ago

Who could object to a measure to eliminate child pornography? Unless you're a fan of it, right? And so the precedent is set - and before you know it, they will be checking your phone for all sorts of things.

matt-attack 5y ago
Well I personally am perfectly fine with several types of child pornography: 1. Drawn or digitally rendered 2. Photos of me personally as a child (bath time, whatnot) I have no problem with photos of me being used by anyone who is unfortunate enough to be born with an attraction to children. Who on this planet can say that I don’t have the right to distribute my own photos of myself?
- aaomidi 5y ago
  > Photos of me personally as a child (bath time, whatnot) This is not CP. Although because there is a human factor to this, I wouldn't be surprised if people got harassed over it. Nudist families are well aware of how shitty NGOs and GOs deal with nudity.

simonblack 5y ago

More and more Government interference that harms company sales. Why buy an Apple? Get some other one instead. Nobody will want even the slightest chance of a false-positive ruining their reputation.

ttflee 5y ago

This type of system is just ready-to-use for a totalitarian regime with minor configuration modifications.

cardosof 5y ago

We traded away privacy for "free" apps and services and now we will also give away privacy to help noble causes. Ok, I just hope the loss of privacy does not cause any harm.

xtiansimon 5y ago

What could go wrong? https://news.ycombinator.com/item?id=27696369 https://news.ycombinator.com/item?id=27696369

john579 5y ago

They are scanning phones for porn material they can use for profit, blackmail, and their own perverted entertainment.