56 ms·

The cite even researchers as leading to the decision. Maybe flaws with perceptual hashes? It took a mere day to discover a hash collision after reverse engineer

by jug 5y ago

The cite even researchers as leading to the decision. Maybe flaws with perceptual hashes? It took a mere day to discover a hash collision after reverse engineering the system. Granted, the colliding image looked like a grey mess, but I wonder how far a malicious party might get after a year.

mlac 5y ago
I don't really understand the argument that was thrown out by many people: "Attackers will frame people by planting hash collisions on their phones!" So... a) a bad actor is going to target someone, and have the resources to generate enough collisions that(b) look like CP, but aren't CP? but are close enough (c) to pass human review and cause an investigation so (d) they need the hash collisions to look like CP, but not be real CP? or ???? If a bad actor wants to frame someone, it's easy to do this today - hack their system or home network, open it to the internet, place the photos, call the FBI and report an anonymous tip, with the URL where it is hosted and open to the internet. Don't need hash collisions. Hacking someone's iPhone (How do you get the photos on there without forensic logs that they were added?) or iCloud so that you can place hash collisions that look like crap and fail to pass the review doesn't make sense and leaves too much of a trail. Oh? And someone won't notice thousands of photos just added to their phone? A bigger threat would be deepfake CP. When that becomes a reality, it will be a mess, because an attacker could theoretically generate an unlimited amount of it, and it will be extremely difficult to tell if it is authentic. Those hashes wouldn't be in the CSAM database, but if the attacker put them out on a server to be taken down, they would get added eventually and then show up in a scan elsewhere. But I'd be pretty horrified (and definitely call my attorney, Apple, and local FBI field office) if thousands of CSAM images just showed up in my iPhone photo stream. Edit: Downvotes are fine, and I completely understand other arguments against this, but this one has just not made sense to me...
- gunapologist99 5y ago
  This is a very simple attack to pull off; email someone with the colliding image embedded in the MIME document. Set height and width equal to zero or one pixel. Even if the user never actually sees it or marks it as spam, it's still on their device and in their email. For bonus points, spoof the "From" as from Apple. At the very least, they're getting SWATted.
  - sweetheart 5y ago
    If this were a reasonable vector, it'd already be an issue, at least if you use large email clients like Gmail, which already scan for CSAM. Edit: Downvotes because?...
    - speeder 5y ago
      Because it IS already an issue. For example when CIA/NSA tools leaked, one of them had precisely this purpose.
    - cmsj 5y ago
      Downvotes because nerds love to construct a super super unlikely corner case to argue against something ;)
      - sweetheart 5y ago
        I'd call that out as totally unfair... if I haven't found myself doing that many times before hahaha.
      - CamperBob2 5y ago
        We, the nerds, only have to be wrong once in order to allow immense harm to occur. Apple voluntarily put itself in that position. Conversely, the governments of the world get to keep trying, over and over.
    - brigade 5y ago
      Those use a different perceptual hash that's intended to be kept secret. It's significantly harder to develop collisions for an algorithm you cannot inspect or obtain the output of. (well also, emailing actual CSAM is way easier and mostly just gets the sender reported)
      - foldr 5y ago
        Surely the sender is also likely to get reported if they email the image with the perceptual hash collision. Lots of companies run these scans. E.g. if you send the image via gmail it will most likely be scanned by Google.
- sweetheart 5y ago
  I also don't understand the vector everyone seems worried about. Also considering that perceptual hashing isn't new, as far as I'm aware, and it hasn't seemingly yet led to any sort of wave of innocent folks being screwed over sham illicit images. I think there _is_ an argument to be made about a system like this being used to track the spread of political material, and it's easy to see how such a system would be terrible for anyone trying to hide from an authoritarian power of some type, but that'd already require Apple is absolutely and completely compromised by an authoritarian regime, which isn't high on my list of concerns.
  - azalemeth 5y ago
    The main issue is that the system is basically a bit of snitching spyware dolled up with cryptographic magic to make it only work with really bad stuff, honest. As recently posted on HN [1] one should be very wary of backdoors, no matter how much one believes only the right guys could ever use them. Once they're there, they're there: it's arrogant beyond believe to think that opponents of yours won't exploit them. I won't use an apple product with this feature. I've been using apple's ecosystem since about 1994. [1] https://twitter.com/matthew_d_green/status/1433470109742518273 https://twitter.com/matthew_d_green/status/14334701097425182... and https://news.ycombinator.com/item?id=28404219 https://news.ycombinator.com/item?id=28404219
    - mlac 5y ago
      I understand your points on backdoors, but I don't understand people trying to argue that the "Cryptographic Magic" is the bad part that will send people to prison due immediately upon "HASH COLLISIONS", in my view, takes away from the actual concerning points.
      - Zak 5y ago
        What most people are focused on is that this is spyware built in to the operating system. The fact that perceptual hashes have some weak points could also lead to problems, but that's not the main focus of most opposition to this system.
mbesto 5y ago
They also could have had a legitimate claim to non-technical people (at the FBI/NSA/blah) that "you sure you want this? this system will be abused...someone could write a hash collision". The Gov't agency scoffed because they didn't believe it. So Apple called them out on it by letting the market decide. They now have that evidence.
- notriddle 5y ago
  Never attribute to 4D chess what can be adequately explained by knee-jerk reaction.
  - mortenjorck 5y ago
    This is an excellent and perceptive Hanlon corollary.
    - ksm1717 5y ago
      Never attribute credence to an argument because it has a name
      - gfodor 5y ago
        Razors are inherently self-identifying as heuristics - not rules. So no need to add more disclaimers.
      - ZainRiz 5y ago
        Never attribute credence to an argument because it was stated confidently
chillfox 5y ago
The colliding images stopped looking like a grey mess within the week and just looked like a normal low quality image.

by jug 5y ago

mlac 5y ago

I don't really understand the argument that was thrown out by many people: "Attackers will frame people by planting hash collisions on their phones!" So... a) a bad actor is going to target someone, and have the resources to generate enough collisions that(b) look like CP, but aren't CP? but are close enough (c) to pass human review and cause an investigation so (d) they need the hash collisions to look like CP, but not be real CP? or ???? If a bad actor wants to frame someone, it's easy to do this today - hack their system or home network, open it to the internet, place the photos, call the FBI and report an anonymous tip, with the URL where it is hosted and open to the internet. Don't need hash collisions. Hacking someone's iPhone (How do you get the photos on there without forensic logs that they were added?) or iCloud so that you can place hash collisions that look like crap and fail to pass the review doesn't make sense and leaves too much of a trail. Oh? And someone won't notice thousands of photos just added to their phone? A bigger threat would be deepfake CP. When that becomes a reality, it will be a mess, because an attacker could theoretically generate an unlimited amount of it, and it will be extremely difficult to tell if it is authentic. Those hashes wouldn't be in the CSAM database, but if the attacker put them out on a server to be taken down, they would get added eventually and then show up in a scan elsewhere. But I'd be pretty horrified (and definitely call my attorney, Apple, and local FBI field office) if thousands of CSAM images just showed up in my iPhone photo stream. Edit: Downvotes are fine, and I completely understand other arguments against this, but this one has just not made sense to me...

gunapologist99 5y ago
This is a very simple attack to pull off; email someone with the colliding image embedded in the MIME document. Set height and width equal to zero or one pixel. Even if the user never actually sees it or marks it as spam, it's still on their device and in their email. For bonus points, spoof the "From" as from Apple. At the very least, they're getting SWATted.
- sweetheart 5y ago
  If this were a reasonable vector, it'd already be an issue, at least if you use large email clients like Gmail, which already scan for CSAM. Edit: Downvotes because?...
  - speeder 5y ago
    Because it IS already an issue. For example when CIA/NSA tools leaked, one of them had precisely this purpose.
  - cmsj 5y ago
    Downvotes because nerds love to construct a super super unlikely corner case to argue against something ;)
    - sweetheart 5y ago
      I'd call that out as totally unfair... if I haven't found myself doing that many times before hahaha.
    - CamperBob2 5y ago
      We, the nerds, only have to be wrong once in order to allow immense harm to occur. Apple voluntarily put itself in that position. Conversely, the governments of the world get to keep trying, over and over.
  - brigade 5y ago
    Those use a different perceptual hash that's intended to be kept secret. It's significantly harder to develop collisions for an algorithm you cannot inspect or obtain the output of. (well also, emailing actual CSAM is way easier and mostly just gets the sender reported)
    - foldr 5y ago
      Surely the sender is also likely to get reported if they email the image with the perceptual hash collision. Lots of companies run these scans. E.g. if you send the image via gmail it will most likely be scanned by Google.
sweetheart 5y ago
I also don't understand the vector everyone seems worried about. Also considering that perceptual hashing isn't new, as far as I'm aware, and it hasn't seemingly yet led to any sort of wave of innocent folks being screwed over sham illicit images. I think there _is_ an argument to be made about a system like this being used to track the spread of political material, and it's easy to see how such a system would be terrible for anyone trying to hide from an authoritarian power of some type, but that'd already require Apple is absolutely and completely compromised by an authoritarian regime, which isn't high on my list of concerns.
- azalemeth 5y ago
  The main issue is that the system is basically a bit of snitching spyware dolled up with cryptographic magic to make it only work with really bad stuff, honest. As recently posted on HN [1] one should be very wary of backdoors, no matter how much one believes only the right guys could ever use them. Once they're there, they're there: it's arrogant beyond believe to think that opponents of yours won't exploit them. I won't use an apple product with this feature. I've been using apple's ecosystem since about 1994. [1] https://twitter.com/matthew_d_green/status/1433470109742518273 https://twitter.com/matthew_d_green/status/14334701097425182... and https://news.ycombinator.com/item?id=28404219 https://news.ycombinator.com/item?id=28404219
  - mlac 5y ago
    I understand your points on backdoors, but I don't understand people trying to argue that the "Cryptographic Magic" is the bad part that will send people to prison due immediately upon "HASH COLLISIONS", in my view, takes away from the actual concerning points.
    - Zak 5y ago
      What most people are focused on is that this is spyware built in to the operating system. The fact that perceptual hashes have some weak points could also lead to problems, but that's not the main focus of most opposition to this system.

mbesto 5y ago

They also could have had a legitimate claim to non-technical people (at the FBI/NSA/blah) that "you sure you want this? this system will be abused...someone could write a hash collision". The Gov't agency scoffed because they didn't believe it. So Apple called them out on it by letting the market decide. They now have that evidence.

notriddle 5y ago
Never attribute to 4D chess what can be adequately explained by knee-jerk reaction.
- mortenjorck 5y ago
  This is an excellent and perceptive Hanlon corollary.
  - ksm1717 5y ago
    Never attribute credence to an argument because it has a name
    - gfodor 5y ago
      Razors are inherently self-identifying as heuristics - not rules. So no need to add more disclaimers.
    - ZainRiz 5y ago
      Never attribute credence to an argument because it was stated confidently

chillfox 5y ago

The colliding images stopped looking like a grey mess within the week and just looked like a normal low quality image.