7 ms·

The better solution is to have the image hashed and NOT sent to facebook. If the hash matches an existing image [A]. If the hash isn't found [B]. [A]: human ca

The better solution is to have the image hashed and NOT sent to facebook. If the hash matches an existing image [A]. If the hash isn't found [B]. [A]: human can review the image they already have [B]: Facebook waits until someone uploads an image that matches and then reviews the image (as normal) but with a marker alerting the problem. The benefit is that people NOT affected won't have to upload lots of images of themselves to facebook personal for "review" just to be sure. The problem is that should facebook update it's hashing or find better ways to match images, having the original image would allow them to transition. This point is moot though since facebook claims not to save the image anyway.

pishpash 9y ago
You'll have to guarantee the authenticity of the client side code, but yes. Here's what you do: on the client side embed image into a semantic space (using NN or whatever), quantize, _then_ hash the representation. Afterwards you send the hash. If you had to change client side code, you can just ask the user to redo the process.
- Xeoncross 9y ago
  Yes, I was hoping they would not simply `sha1()` the image data. I wouldn't worry much about trying to fake hashes since we are already allowing them to provide whatever image as input they want (hence the review).
ghshephard 9y ago
The obvious reason why Facebook didn't go down this path is one of scale - It's easy (and altogether likely) for an attacker to pollute the hashes with innocuous pictures. But, even more importantly, once you reveal your hashing system, it's relatively trivial for people to work around them and modify the pictures in such a way as to render the hashing system useless. Has to be done server side for these reasons.
- beager 9y ago
  If “scale” is the reason they insist that explicit images be uploaded directly to them instead of hashed beforehand, then they really don’t care about this program enough to do it right. They can easily scale this by allowing hash uploads, using a trivial client side classifier to guess at whether an image is actually served well by this technology, and confirm positives with CV after a bad actor reposts their media on the site, with human intervention only on the fringes.
- theptip 9y ago
  Could you elaborate why you think it's easy to pollute hashes/fingerprints in this scheme? It seems to me that it would be hard to do reliably, since each hash submitted is going to be linked to an individual account. If an account submitted a bunch of hashes that matched non-explicit images, all further hash submissions from that user account could be rejected. (In the GP's proposed scheme a human can still verify a sample of all positive matches, to validate if they are true or false positives). I think the point about revealing your fingerprinting system is a good one. But I wonder if the equilibrium point would still be better than what we have now, i.e. unsophisticated posters would get caught, and sophisticated ones might not, vs. what we have now where there is no way to catch any of these posts.
rcpt 9y ago
Kind of weird that they didn't do things this way as they supposedly already use hashing to flag child porn cf. http://www.wired.co.uk/article/iwf-hash-lists-child-abuse-images http://www.wired.co.uk/article/iwf-hash-lists-child-abuse-im...
- Dylan16807 9y ago
  That system works the same way as what they're implementing. Someone reviews the raw image and enters the hash into the system.
  - JauntyHatAngle 9y ago
    >Someone reviews the raw image and enters the hash into the system. Now that is a job I really would not want to have. Talk about mentally scarring.
    - reificator 9y ago
      Isn't that one of the highest turnover jobs that exists? It's really hard to keep people coming into work when their work is to review twisted content all day.
      - ben_w 9y ago
        I have a hypothesis that undiscovered child fanciers will seek jobs like this. I also hypothesise (on the basis of news articles saying how low the reoffending rates are compared to other categories of offender) that the distribution of sadistic narcissism is the same or similar for them as it is everyone else, so when they sit down at the office desk and see a screaming child, most of them will hate themselves if they didn’t already. I’m not so sure about the second hypothesis though. It’s based on what newspapers say, and I’ve learned to distrust them.
trjordan 9y ago
There was an excellent Twitter thread on why they didn't do this: https://twitter.com/alexstamos/status/928740488395608065 https://twitter.com/alexstamos/status/928740488395608065 The salient bit: > A quick note to everybody who says 'calculate the hash locally': > A) Photo fingerprinting algorithms are usually not included in clients to prevent the development of circumvention techniques. > B) Humans need to review to prevent adversarial reporting. Further down the thread, they discuss further that PhotoDNA and similar algorithms are, in particular, not resistant to being tricked. Until they are, it's another layer to prevent adversaries from fully decomposing how they're hashing it, which is a good thing. It's a trade-off, for sure, but this is a hard problem with no cut and dry answer. Facebook already has the images in question, because you have to be sending them to somebody in order to mark them as NCII. Asking yet another Facebook service to look at them without storing them seems reasonable to me. You may disagree.
- valuearb 9y ago
  It's not a hard problem. Both A & B are bogus reasons. A) Any hash is going to be easily circumvented by anyone who owns Photoshop or any image editor. It's only going to catch those dumb enough not to modify the images, ie. most everyone. Doing the hash server side isn't the big problem though, because if you trust the Facebook App then by default you should trust the Facebook web applications. B) The big problem is you take care of adversarial reporting at the end, not the beginning. If they get a hit on a reported hash, only then do they tell the photo owner and ask for proof that it's their image. It's bonkers that Facebook expects people terrified of their photos being circulated to send them those images so a Facebook employee can eye fondle them. This is just par for the course for Facebook, as a business they are so tone death and oblivious to basic human behavior and feelings that it seems like the entire company is infected with the same alleged autism of it's founder.
  - kelnos 9y ago
    A) I think we're talking about modification-resistant fingerprinting methods, not just a simple hash of the data. B) That's a support nightmare; it would be nearly trivial to start submitting false reports from a ton of accounts without establishing a pattern so those accounts could easily be ignored. Regardless, how do you prove that you "own an image" (what does that even mean?) and that it was reported as an "intimate photo" maliciously? I agree that FB's current solution is really bad, and if it were me, I certainly wouldn't pro-actively send in photos of myself, but I'm at a loss for how they can address A & B.

pishpash 9y ago

You'll have to guarantee the authenticity of the client side code, but yes. Here's what you do: on the client side embed image into a semantic space (using NN or whatever), quantize, _then_ hash the representation. Afterwards you send the hash. If you had to change client side code, you can just ask the user to redo the process.

Xeoncross 9y ago
Yes, I was hoping they would not simply `sha1()` the image data. I wouldn't worry much about trying to fake hashes since we are already allowing them to provide whatever image as input they want (hence the review).

ghshephard 9y ago

The obvious reason why Facebook didn't go down this path is one of scale - It's easy (and altogether likely) for an attacker to pollute the hashes with innocuous pictures. But, even more importantly, once you reveal your hashing system, it's relatively trivial for people to work around them and modify the pictures in such a way as to render the hashing system useless. Has to be done server side for these reasons.

beager 9y ago
If “scale” is the reason they insist that explicit images be uploaded directly to them instead of hashed beforehand, then they really don’t care about this program enough to do it right. They can easily scale this by allowing hash uploads, using a trivial client side classifier to guess at whether an image is actually served well by this technology, and confirm positives with CV after a bad actor reposts their media on the site, with human intervention only on the fringes.
theptip 9y ago
Could you elaborate why you think it's easy to pollute hashes/fingerprints in this scheme? It seems to me that it would be hard to do reliably, since each hash submitted is going to be linked to an individual account. If an account submitted a bunch of hashes that matched non-explicit images, all further hash submissions from that user account could be rejected. (In the GP's proposed scheme a human can still verify a sample of all positive matches, to validate if they are true or false positives). I think the point about revealing your fingerprinting system is a good one. But I wonder if the equilibrium point would still be better than what we have now, i.e. unsophisticated posters would get caught, and sophisticated ones might not, vs. what we have now where there is no way to catch any of these posts.

rcpt 9y ago

Kind of weird that they didn't do things this way as they supposedly already use hashing to flag child porn cf. http://www.wired.co.uk/article/iwf-hash-lists-child-abuse-images http://www.wired.co.uk/article/iwf-hash-lists-child-abuse-im...

Dylan16807 9y ago
That system works the same way as what they're implementing. Someone reviews the raw image and enters the hash into the system.
- JauntyHatAngle 9y ago
  >Someone reviews the raw image and enters the hash into the system. Now that is a job I really would not want to have. Talk about mentally scarring.
  - reificator 9y ago
    Isn't that one of the highest turnover jobs that exists? It's really hard to keep people coming into work when their work is to review twisted content all day.
    - ben_w 9y ago
      I have a hypothesis that undiscovered child fanciers will seek jobs like this. I also hypothesise (on the basis of news articles saying how low the reoffending rates are compared to other categories of offender) that the distribution of sadistic narcissism is the same or similar for them as it is everyone else, so when they sit down at the office desk and see a screaming child, most of them will hate themselves if they didn’t already. I’m not so sure about the second hypothesis though. It’s based on what newspapers say, and I’ve learned to distrust them.

trjordan 9y ago

There was an excellent Twitter thread on why they didn't do this: https://twitter.com/alexstamos/status/928740488395608065 https://twitter.com/alexstamos/status/928740488395608065 The salient bit: > A quick note to everybody who says 'calculate the hash locally': > A) Photo fingerprinting algorithms are usually not included in clients to prevent the development of circumvention techniques. > B) Humans need to review to prevent adversarial reporting. Further down the thread, they discuss further that PhotoDNA and similar algorithms are, in particular, not resistant to being tricked. Until they are, it's another layer to prevent adversaries from fully decomposing how they're hashing it, which is a good thing. It's a trade-off, for sure, but this is a hard problem with no cut and dry answer. Facebook already has the images in question, because you have to be sending them to somebody in order to mark them as NCII. Asking yet another Facebook service to look at them without storing them seems reasonable to me. You may disagree.

valuearb 9y ago
It's not a hard problem. Both A & B are bogus reasons. A) Any hash is going to be easily circumvented by anyone who owns Photoshop or any image editor. It's only going to catch those dumb enough not to modify the images, ie. most everyone. Doing the hash server side isn't the big problem though, because if you trust the Facebook App then by default you should trust the Facebook web applications. B) The big problem is you take care of adversarial reporting at the end, not the beginning. If they get a hit on a reported hash, only then do they tell the photo owner and ask for proof that it's their image. It's bonkers that Facebook expects people terrified of their photos being circulated to send them those images so a Facebook employee can eye fondle them. This is just par for the course for Facebook, as a business they are so tone death and oblivious to basic human behavior and feelings that it seems like the entire company is infected with the same alleged autism of it's founder.
- kelnos 9y ago
  A) I think we're talking about modification-resistant fingerprinting methods, not just a simple hash of the data. B) That's a support nightmare; it would be nearly trivial to start submitting false reports from a ton of accounts without establishing a pattern so those accounts could easily be ignored. Regardless, how do you prove that you "own an image" (what does that even mean?) and that it was reported as an "intimate photo" maliciously? I agree that FB's current solution is really bad, and if it were me, I certainly wouldn't pro-actively send in photos of myself, but I'm at a loss for how they can address A & B.