5 ms·

There's been a lot of focus on the likelihood of collisions and whether someone could upload eg; an image with a matching hash to your device to "set you up", e

There's been a lot of focus on the likelihood of collisions and whether someone could upload eg; an image with a matching hash to your device to "set you up", etc. But what's still extremely concerning is that there is still no guarantee that the hash list used can't be coopted for another purpose (eg; politically insensitive content).

criticaltinker 5y ago
The OP mentions that two countries have to agree to add a file to the list, but your concern is definitely valid: > Perhaps the most concerning part of the whole scheme is the database itself. Since the original images are (understandably) not available for inspection, it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse.
- ulzeraj 5y ago
  What if those two countries can be Poland and Hungary? These two countries have been passing lots of laws to ostracize and criminalize pro-LGBT content and are friendly to each other.
  - tehnub 5y ago
    Apple did mention in their security thread model document [0] this: Apple will also refuse all requests to instruct human reviewers to file reports for anything other than CSAM materials for accounts that exceed the match threshold. [0]: https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model...
    - nybble41 5y ago
      That's too little, too late. By the time any human reviewer can evaluate whether the images should have been in the database the encryption on the backups has already been circumvented and other parties have already been given access to your private files.
      - shadowfiend 5y ago
        That is not how this works. Please read up on the functioning of the system before chiming in with such certainty on its behavior. The only thing they will have gained access to are the “derivatives” (presumably lower res versions) of the matched photos, which if this is done to frame you is strictly the fake CSAM.
        nybble41 5y ago
        I'm aware of how the system works. Those are still your private files which were supposed to be encrypted and which were revealed (either partially or fully, makes no difference) to another party. That fact that this review process is even possible means that the encryption on the backups can't be trusted.
  - tester756 5y ago
    > These two countries have been passing lots of laws to criminalize pro-LGBT content What do you mean I live in one of those countries and I'd want to be aware
    - vineyardmike 5y ago
      https://googlethatforyou.com?q=poland%20anti-lgbt https://googlethatforyou.com?q=poland%20anti-lgbt but actually this is a good starting point: https://en.wikipedia.org/wiki/LGBT_rights_in_Poland https://en.wikipedia.org/wiki/LGBT_rights_in_Poland
  - WesolyKubeczek 5y ago
    Fortunately, Hungary and Poland, even combined, are quite small a fish for Apple to just tell them to go pound sand in some form or other. They can even ban iPhone sales, people will just buy them in Czechia. It's not like China, or India, who not only have huge markets, but could easily hold a chunk of Apple's supply chain hostage. It's very easy to uphold human rights if it doesn't actually cost you anything.
- JoshTko 5y ago
  It's probably relatively trivial for the US or China to coerce another county to agree to an image.
cat199 5y ago
this also punts the debate to the checking process instead of the fact that there even is a process to start with..
- version_five 5y ago
  Yes I was about to say the same thing. Hash collisions are an extra concern about what apple is doing, but even if they were as collision free as cryptographic hashes, that would not make the invasion of privacy OK. The technical discussion is something that apple can easily parry and is the wrong framing of this debate.
- rustymonday 5y ago
  I'm so glad you said this. As someone not in the tech field, it is incredibly concerning that half the people here on Hacker News, people who help build this kind of technology, do not seem to be concerned with what Apple is doing.
bequanna 5y ago
> there is still no guarantee that the hash list used can't be coopted for another purpose (eg; politically insensitive content). That isn't a bug, it is a feature and will be the main use of this functionality. The "preventing child pornography" reasoning was specifically chosen so that Apple could openly coordinate with governments to violate your privacy while avoiding criticism.
Someone 5y ago
There wasn’t any guarantee that Apple didn’t have and use such technology before they made this feature public, or even already was running it in the current version of iOS. If you trusted Apple not to stealthily run such technology before, the question is how much less (if any) you trust them now. If you didn’t, I don’t think anything changed.
rovr138 5y ago
On top of that, what happens if a court/government orders them to give them all the current data about people with matches, regardless of the 30 matches. They can't say if it is or not a match so they have to go after the individuals. Is that enough evidence for a warrant? Someone in the court thinks it's true and can't prosecute?, oh, it got leaked. -- Not every country has the same protections about innocent until proven guilty. And even then, we've seen cases in the US where someone has been held in jail indefinitely until they provide a password, - https://arstechnica.com/tech-policy/2016/04/child-porn-suspect-jailed-for-7-months-for-refusing-to-decrypt-hard-drives/ https://arstechnica.com/tech-policy/2016/04/child-porn-suspe... - https://nakedsecurity.sophos.com/2016/04/28/suspect-who-wont-decrypt-hard-drives-jailed-indefinitely/ https://nakedsecurity.sophos.com/2016/04/28/suspect-who-wont...
- int_19h 5y ago
  More broadly speaking, every part of this scheme that is currently an arbitrary Apple decision (and not a technological limitation), can easily become an arbitrary government decision. And yes, it's true that the governments could always mandate such scanning before. The difference is that it'll be much harder politically for Apple to push back against tweaks to the scheme (such as lowering the bar for manual review / notification of authorities) if they already have it rolled out successfully and publicly argued that it's acceptable in principle, as opposed to pushing back against any kind of scanning at all. Once you establish that something is okay in principle, the specifics can be haggled over. I mean, just imagine this conversation in a Congressional hearing: "So, you only report if there are 30+ CSAM images found by the scan. Does this mean that pedophiles with 20 CSAM images on their phones are not reported?" "Well... yes." "And how did you decide that 30 is the appropriate number? Why not 20, or 10? Do you maybe think that going after CSAM is not that important, after all?" There's a very old joke along these lines that seems particularly appropriate here: "Churchill: Madam, would you sleep with me for five million pounds? Socialite: My goodness, Mr. Churchill… Well, I suppose… we would have to discuss terms, of course… Churchill: Would you sleep with me for five pounds? Socialite: Mr. Churchill, what kind of woman do you think I am?! Churchill: Madam, we’ve already established that. Now we are haggling about the price." Apple has put itself in the position where, from now on, they'll be haggling about the price - and they don't really have much leverage there.

criticaltinker 5y ago

The OP mentions that two countries have to agree to add a file to the list, but your concern is definitely valid: > Perhaps the most concerning part of the whole scheme is the database itself. Since the original images are (understandably) not available for inspection, it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse.

ulzeraj 5y ago
What if those two countries can be Poland and Hungary? These two countries have been passing lots of laws to ostracize and criminalize pro-LGBT content and are friendly to each other.
- tehnub 5y ago
  Apple did mention in their security thread model document [0] this: Apple will also refuse all requests to instruct human reviewers to file reports for anything other than CSAM materials for accounts that exceed the match threshold. [0]: https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model...
  - nybble41 5y ago
    That's too little, too late. By the time any human reviewer can evaluate whether the images should have been in the database the encryption on the backups has already been circumvented and other parties have already been given access to your private files.
    - shadowfiend 5y ago
      That is not how this works. Please read up on the functioning of the system before chiming in with such certainty on its behavior. The only thing they will have gained access to are the “derivatives” (presumably lower res versions) of the matched photos, which if this is done to frame you is strictly the fake CSAM.
      - nybble41 5y ago
        I'm aware of how the system works. Those are still your private files which were supposed to be encrypted and which were revealed (either partially or fully, makes no difference) to another party. That fact that this review process is even possible means that the encryption on the backups can't be trusted.
- tester756 5y ago
  > These two countries have been passing lots of laws to criminalize pro-LGBT content What do you mean I live in one of those countries and I'd want to be aware
  - vineyardmike 5y ago
    https://googlethatforyou.com?q=poland%20anti-lgbt https://googlethatforyou.com?q=poland%20anti-lgbt but actually this is a good starting point: https://en.wikipedia.org/wiki/LGBT_rights_in_Poland https://en.wikipedia.org/wiki/LGBT_rights_in_Poland
- WesolyKubeczek 5y ago
  Fortunately, Hungary and Poland, even combined, are quite small a fish for Apple to just tell them to go pound sand in some form or other. They can even ban iPhone sales, people will just buy them in Czechia. It's not like China, or India, who not only have huge markets, but could easily hold a chunk of Apple's supply chain hostage. It's very easy to uphold human rights if it doesn't actually cost you anything.
JoshTko 5y ago
It's probably relatively trivial for the US or China to coerce another county to agree to an image.

cat199 5y ago

this also punts the debate to the checking process instead of the fact that there even is a process to start with..

version_five 5y ago
Yes I was about to say the same thing. Hash collisions are an extra concern about what apple is doing, but even if they were as collision free as cryptographic hashes, that would not make the invasion of privacy OK. The technical discussion is something that apple can easily parry and is the wrong framing of this debate.
rustymonday 5y ago
I'm so glad you said this. As someone not in the tech field, it is incredibly concerning that half the people here on Hacker News, people who help build this kind of technology, do not seem to be concerned with what Apple is doing.

bequanna 5y ago

> there is still no guarantee that the hash list used can't be coopted for another purpose (eg; politically insensitive content). That isn't a bug, it is a feature and will be the main use of this functionality. The "preventing child pornography" reasoning was specifically chosen so that Apple could openly coordinate with governments to violate your privacy while avoiding criticism.

Someone 5y ago

There wasn’t any guarantee that Apple didn’t have and use such technology before they made this feature public, or even already was running it in the current version of iOS. If you trusted Apple not to stealthily run such technology before, the question is how much less (if any) you trust them now. If you didn’t, I don’t think anything changed.

rovr138 5y ago

On top of that, what happens if a court/government orders them to give them all the current data about people with matches, regardless of the 30 matches. They can't say if it is or not a match so they have to go after the individuals. Is that enough evidence for a warrant? Someone in the court thinks it's true and can't prosecute?, oh, it got leaked. -- Not every country has the same protections about innocent until proven guilty. And even then, we've seen cases in the US where someone has been held in jail indefinitely until they provide a password, - https://arstechnica.com/tech-policy/2016/04/child-porn-suspect-jailed-for-7-months-for-refusing-to-decrypt-hard-drives/ https://arstechnica.com/tech-policy/2016/04/child-porn-suspe... - https://nakedsecurity.sophos.com/2016/04/28/suspect-who-wont-decrypt-hard-drives-jailed-indefinitely/ https://nakedsecurity.sophos.com/2016/04/28/suspect-who-wont...

int_19h 5y ago
More broadly speaking, every part of this scheme that is currently an arbitrary Apple decision (and not a technological limitation), can easily become an arbitrary government decision. And yes, it's true that the governments could always mandate such scanning before. The difference is that it'll be much harder politically for Apple to push back against tweaks to the scheme (such as lowering the bar for manual review / notification of authorities) if they already have it rolled out successfully and publicly argued that it's acceptable in principle, as opposed to pushing back against any kind of scanning at all. Once you establish that something is okay in principle, the specifics can be haggled over. I mean, just imagine this conversation in a Congressional hearing: "So, you only report if there are 30+ CSAM images found by the scan. Does this mean that pedophiles with 20 CSAM images on their phones are not reported?" "Well... yes." "And how did you decide that 30 is the appropriate number? Why not 20, or 10? Do you maybe think that going after CSAM is not that important, after all?" There's a very old joke along these lines that seems particularly appropriate here: "Churchill: Madam, would you sleep with me for five million pounds? Socialite: My goodness, Mr. Churchill… Well, I suppose… we would have to discuss terms, of course… Churchill: Would you sleep with me for five pounds? Socialite: Mr. Churchill, what kind of woman do you think I am?! Churchill: Madam, we’ve already established that. Now we are haggling about the price." Apple has put itself in the position where, from now on, they'll be haggling about the price - and they don't really have much leverage there.