3 ms·

What will happen is that criminals who actually had bad intent will move off of the platform and the ones who get the brunt of the blame are the innocent who ha

by 1-6 5y ago

What will happen is that criminals who actually had bad intent will move off of the platform and the ones who get the brunt of the blame are the innocent who had no ill intent.

vsareto 5y ago
Fairly sure criminals can't prove the efficient market hypothesis either. You still want to catch the dumb ones.
mlac 5y ago
The process is described above, but it’s very hard to “innocently” end up with one of those images that they are looking for from the database. And the way it’s being done (hashes), a collision is highly unlikely. If it does occur it doesn’t mean it’s similar in nature (e.g. innocent picture of own child in bath). The hash isn’t looking at the image content in the sense of “what’s in the picture”, just the bits of the file. So it’s highly, highly unlikely, even if a collision occurs, that the collision would be an image that happens to be another child innocently bathing.
- gojomo 5y ago
  > but it’s very hard to “innocently” end up with one of those images that they are looking for from the database. Are you sure, given how many iPhone takeover/jailbreak bugs regularly exist, including the recent 0-click iMessage bug? Would you like to dare a hacking group, domestic or foreign, to land a single verboten image on your iPhone? > And the way it’s being done (hashes), a collision is highly unlikely. The limited details so far are suggestive that its using perceptual hashes, which are more susceptible to collision-engineering/false-positives than cryptographically-secure hashes.
- aviraldg 5y ago
  Presumably the verification that these false matches are not problematic is manual? That's not good enough given how our media currently works. Imagine articles published when information about such a check leaks that say "celebrity X's phone checked by police for suspected CSAM." While that is the truth ("suspected") no one cares about that nuance and such a person would get cancelled very quickly, even if there was no evidence of wrongdoing.
- ihattendorf 5y ago
  What happens if someone spams CP via iMessage to their enemies? Additionally, this isn't just a hash of the file but a perceptual hash on the image content. So e.g. changing a single bit in the image would create a different cryptographic hash, but generally not a different perceptual hash.
- alasano 5y ago
  Furthermore, "highly unlikely" is an understatement. Using SHA256 there are 2^256 hashes. That's 115792089237316195423570985008687907853269984665640564039457584007913129639936. Probability of collision (two different inputs generating the same hash) is infinitesimal.
- elzbardico 5y ago
  It would be trivial for a bad intentioned actor to send those images to a target that he wanted to incriminate if the target is not tech savvy.
- heavyset_go 5y ago
  > The process is described above, but it’s very hard to “innocently” end up with one of those images that they are looking for from the database. Actually, it's very easy to end up with an image that has a similar perceptual hash to an illegal image. They are not doing MD5 hashing, they're taking perceptual hashes and then using something like the hamming distance or Levenshtein distance to make a fuzzy match with hashes from illegal images. I've built products using these methods, and it is incredibly easy to make a fuzzy match based on perceptual hashes from two images that have nothing to do with each other.
- curryst 5y ago
  Presuming that is what they're doing, it won't stay that way for long. It's absurdly easy to change the hash of a file. In the case of something like JPEG, you don't even have to change the file itself, you can just change the metadata. Apple could presumably only hash the image itself, but again, all you have to do is make tiny, imperceptible to humans changes to the image and the hash is totally different. Long story short, this is either nearly pointless and privacy invasive, or it's about to get drastically more invasive to be effective.
- JohnFen 5y ago
  > it’s very hard to “innocently” end up with one of those images that they are looking for from the database. Of course, there's no telling what images will be in the database when this inevitably expands beyond CP.
1-6 5y ago
I actually do want a feature where I will get a notification on my phone if someone nearby has snagged a picture of me or my kids. A creepy shopper at Costco once took a pic of my 2-year-old. They pretended to be working up a conversation with my little kid meanwhile they had their phone at hip-level to snag a shot. I even confronted them about it. The Costco employees couldn't do a thing. Phones with cameras are useful but have also become a nuisance. Edit: It makes sense that such a notification system would be impossible to implement. Perhaps, registered offenders should not only have their location placed on a map but their phones have certain privileges disabled or monitored.
- jtbayly 5y ago
  That's silly. You're in public. Your photo (and your kids') are being taken all the time.
  - 1-6 5y ago
    It's not the case when you're inside Costco. I'd also argue that it's not open to the public because you're required to have a membership to go in.
    - Spooky23 5y ago
      The point is, Costco is taking many pictures of you, including possibly correlating with events or actions.
      - lp0_on_fire 5y ago
        We accept Costco taking pictures of you as a condition of entering the store and securing a membership. The same does not go for random creeps wanting to take close up pictures of a child.
        Spooky23 5y ago
        In this specific example, sure. You have no expectation of privacy in public and you are photographed, repeatedly. The creep behavior is the problem. The photograph is a symptom of it.
alasano 5y ago
Just lobby Congress to make it illegal to switch to Android. Problem solved.

by 1-6 5y ago

What will happen is that criminals who actually had bad intent will move off of the platform and the ones who get the brunt of the blame are the innocent who had no ill intent.

vsareto 5y ago

Fairly sure criminals can't prove the efficient market hypothesis either. You still want to catch the dumb ones.

mlac 5y ago

The process is described above, but it’s very hard to “innocently” end up with one of those images that they are looking for from the database. And the way it’s being done (hashes), a collision is highly unlikely. If it does occur it doesn’t mean it’s similar in nature (e.g. innocent picture of own child in bath). The hash isn’t looking at the image content in the sense of “what’s in the picture”, just the bits of the file. So it’s highly, highly unlikely, even if a collision occurs, that the collision would be an image that happens to be another child innocently bathing.

gojomo 5y ago
> but it’s very hard to “innocently” end up with one of those images that they are looking for from the database. Are you sure, given how many iPhone takeover/jailbreak bugs regularly exist, including the recent 0-click iMessage bug? Would you like to dare a hacking group, domestic or foreign, to land a single verboten image on your iPhone? > And the way it’s being done (hashes), a collision is highly unlikely. The limited details so far are suggestive that its using perceptual hashes, which are more susceptible to collision-engineering/false-positives than cryptographically-secure hashes.
aviraldg 5y ago
Presumably the verification that these false matches are not problematic is manual? That's not good enough given how our media currently works. Imagine articles published when information about such a check leaks that say "celebrity X's phone checked by police for suspected CSAM." While that is the truth ("suspected") no one cares about that nuance and such a person would get cancelled very quickly, even if there was no evidence of wrongdoing.
ihattendorf 5y ago
What happens if someone spams CP via iMessage to their enemies? Additionally, this isn't just a hash of the file but a perceptual hash on the image content. So e.g. changing a single bit in the image would create a different cryptographic hash, but generally not a different perceptual hash.
alasano 5y ago
Furthermore, "highly unlikely" is an understatement. Using SHA256 there are 2^256 hashes. That's 115792089237316195423570985008687907853269984665640564039457584007913129639936. Probability of collision (two different inputs generating the same hash) is infinitesimal.
elzbardico 5y ago
It would be trivial for a bad intentioned actor to send those images to a target that he wanted to incriminate if the target is not tech savvy.
heavyset_go 5y ago
> The process is described above, but it’s very hard to “innocently” end up with one of those images that they are looking for from the database. Actually, it's very easy to end up with an image that has a similar perceptual hash to an illegal image. They are not doing MD5 hashing, they're taking perceptual hashes and then using something like the hamming distance or Levenshtein distance to make a fuzzy match with hashes from illegal images. I've built products using these methods, and it is incredibly easy to make a fuzzy match based on perceptual hashes from two images that have nothing to do with each other.
curryst 5y ago
Presuming that is what they're doing, it won't stay that way for long. It's absurdly easy to change the hash of a file. In the case of something like JPEG, you don't even have to change the file itself, you can just change the metadata. Apple could presumably only hash the image itself, but again, all you have to do is make tiny, imperceptible to humans changes to the image and the hash is totally different. Long story short, this is either nearly pointless and privacy invasive, or it's about to get drastically more invasive to be effective.
JohnFen 5y ago
> it’s very hard to “innocently” end up with one of those images that they are looking for from the database. Of course, there's no telling what images will be in the database when this inevitably expands beyond CP.

1-6 5y ago

I actually do want a feature where I will get a notification on my phone if someone nearby has snagged a picture of me or my kids. A creepy shopper at Costco once took a pic of my 2-year-old. They pretended to be working up a conversation with my little kid meanwhile they had their phone at hip-level to snag a shot. I even confronted them about it. The Costco employees couldn't do a thing. Phones with cameras are useful but have also become a nuisance. Edit: It makes sense that such a notification system would be impossible to implement. Perhaps, registered offenders should not only have their location placed on a map but their phones have certain privileges disabled or monitored.

jtbayly 5y ago
That's silly. You're in public. Your photo (and your kids') are being taken all the time.
- 1-6 5y ago
  It's not the case when you're inside Costco. I'd also argue that it's not open to the public because you're required to have a membership to go in.
  - Spooky23 5y ago
    The point is, Costco is taking many pictures of you, including possibly correlating with events or actions.
    - lp0_on_fire 5y ago
      We accept Costco taking pictures of you as a condition of entering the store and securing a membership. The same does not go for random creeps wanting to take close up pictures of a child.
      - Spooky23 5y ago
        In this specific example, sure. You have no expectation of privacy in public and you are photographed, repeatedly. The creep behavior is the problem. The photograph is a symptom of it.

alasano 5y ago

Just lobby Congress to make it illegal to switch to Android. Problem solved.