5 ms·

There was an excellent Twitter thread on why they didn't do this: https://twitter.com/alexstamos/status/928740488395608065 https://twitter.com/alexstamos/status

There was an excellent Twitter thread on why they didn't do this: https://twitter.com/alexstamos/status/928740488395608065 https://twitter.com/alexstamos/status/928740488395608065 The salient bit: > A quick note to everybody who says 'calculate the hash locally': > A) Photo fingerprinting algorithms are usually not included in clients to prevent the development of circumvention techniques. > B) Humans need to review to prevent adversarial reporting. Further down the thread, they discuss further that PhotoDNA and similar algorithms are, in particular, not resistant to being tricked. Until they are, it's another layer to prevent adversaries from fully decomposing how they're hashing it, which is a good thing. It's a trade-off, for sure, but this is a hard problem with no cut and dry answer. Facebook already has the images in question, because you have to be sending them to somebody in order to mark them as NCII. Asking yet another Facebook service to look at them without storing them seems reasonable to me. You may disagree.

valuearb 9y ago
It's not a hard problem. Both A & B are bogus reasons. A) Any hash is going to be easily circumvented by anyone who owns Photoshop or any image editor. It's only going to catch those dumb enough not to modify the images, ie. most everyone. Doing the hash server side isn't the big problem though, because if you trust the Facebook App then by default you should trust the Facebook web applications. B) The big problem is you take care of adversarial reporting at the end, not the beginning. If they get a hit on a reported hash, only then do they tell the photo owner and ask for proof that it's their image. It's bonkers that Facebook expects people terrified of their photos being circulated to send them those images so a Facebook employee can eye fondle them. This is just par for the course for Facebook, as a business they are so tone death and oblivious to basic human behavior and feelings that it seems like the entire company is infected with the same alleged autism of it's founder.
- kelnos 9y ago
  A) I think we're talking about modification-resistant fingerprinting methods, not just a simple hash of the data. B) That's a support nightmare; it would be nearly trivial to start submitting false reports from a ton of accounts without establishing a pattern so those accounts could easily be ignored. Regardless, how do you prove that you "own an image" (what does that even mean?) and that it was reported as an "intimate photo" maliciously? I agree that FB's current solution is really bad, and if it were me, I certainly wouldn't pro-actively send in photos of myself, but I'm at a loss for how they can address A & B.
  - pavel_lishin 9y ago
    > That's a support nightmare; it would be nearly trivial to start submitting false reports from a ton of accounts without establishing a pattern so those accounts could easily be ignored. What's stopping me from flooding false reports now? Arguably, only reviewing the image they already have once a hash matches is less work, since they wouldn't have to review all incoming reports, only matched ones.
    - alsetmusic 9y ago
      > Arguably, only reviewing the image they already have once a hash matches is less work, since they wouldn't have to review all incoming reports, only matched ones. They would still need to investigate reports that didn’t match known hashes, but you’re right that this would be beneficial. Images that match a known hash could be prioritized during review.
      - pavel_lishin 9y ago
        Don't they have to do that now anyway?
  - phkahler 9y ago
    The solution is to hash it locally and send the hash to Facebook without saving a local copy. Simple as that. Sure, someone could still do a lot of work to intercept or reverse engineer it and make an app to circumvent the hashing. But even if such an app existed, it would be obscure and people trying to upload unauthorized pics wouldn't bother or even know it exists. If you've got pics you don't want on the internet, the first rule is to not upload them to anywhere on the internet. Actually the real first rule is not to take them. Flash people your bits in person instead.
    - icebraining 9y ago
      Actually the real first rule is not to take them. Flash people your bits in person instead. Assuming you were the one to take the pictures, of course, and not got them from your ex-partner that's trying to pressure you by threatening to publish them on FB.
- yawaramin 9y ago
  Hey, please don't use phrasing like '... infected with ... autism'. First of all, autism isn't an infection, and second it dehumanises people with autism when you associate them with negative connotations like not being aware of human feelings and 'infecting' others. They are people too, don't dump on them.
  - gruez 9y ago
    >First of all, autism isn't an infection it's being used as a rhetorical device. the implication here is that although autism isn't actually being spread per se, the environment at facebook (caused by profit motive or otherwise) causes "tone death and oblivious to basic human behavior and feelings". >second it dehumanises people with autism when you associate them with negative connotations like not being aware of human feelings im aware that autism exists on a spectrum, and there are high functioning autists, but isn't that at generally true?
    - yawaramin 9y ago
      For some people on the spectrum, yes, but you have to realise it's a severely frustrating experience for them as well because they have feelings and want to make human connections too. So it's not helpful to paint them as being oblivious. As I said, it's dehumanising. They deserve dignity and that includes being aware of the language we use about them.
    - scott_s 9y ago
      That it's being used as a rhetorical device is immaterial. It's still dehumanizing. And no, people on the autistic spectrum are fully aware of human feelings. But people on the autistic spectrum typically have difficulty reading other people's emotions.
    - duskwuff 9y ago
      > it's being used as a rhetorical device And it's an ugly, demeaning rhetorical device which insults a large group of people, some of whom are users of this site. (Not that it'd be any more acceptable if they weren't present.) Find another way of expressing yourself.
- pvg 9y ago
  It's definitely a hard problem, especially if you think about it a little rather than just dismissing it out of hand. For one thing, the hashes being discussed are resilient to some modifications - they're not general purpose hashes. More importantly, this is a case in which FB is making a serious attempt to deal with a complex issue rather than simply throwing up their hands and trotting out one of their standard excuses - 'it's not our fault, there isn't anything we can do, it's a side effect of the algorithm, we aren't responsible for how people use the platform' etc. This is a welcome change.
- recursive 9y ago
  > It's only going to catch those dumb enough not to modify the images You seem to be assuming they're using a general purpose hash, rather than one suitable for this purpose. That seems unlikely.
topmonk 9y ago
First, Facebook isn't the only place to share revenge porn, so they aren't really solving the problem, only reducing it slightly. Second, security by obscurity is not a good thing. Not running the algorithm locally, not to mention producing the source to their algorithm is not going to help in the long run. Third, regardless of whether the source of the algorithm is known or not, it would be a simple task to upload a test image of oneself, and then attempt to circumvent the algorithm using dummy accounts. Fourth, if someone abuses the service by uploading the hashes of a benign images, it can be reported at that time. No need to pre-emptively guard against this.
Havoc 9y ago
>Humans need to review uhm...this seems like a bit of a flaw in the plan
amptorn 9y ago
> Humans need to review to prevent adversarial reporting Humans already need to review the takedown reports themselves to prevent adversarial reporting!

valuearb 9y ago

It's not a hard problem. Both A & B are bogus reasons. A) Any hash is going to be easily circumvented by anyone who owns Photoshop or any image editor. It's only going to catch those dumb enough not to modify the images, ie. most everyone. Doing the hash server side isn't the big problem though, because if you trust the Facebook App then by default you should trust the Facebook web applications. B) The big problem is you take care of adversarial reporting at the end, not the beginning. If they get a hit on a reported hash, only then do they tell the photo owner and ask for proof that it's their image. It's bonkers that Facebook expects people terrified of their photos being circulated to send them those images so a Facebook employee can eye fondle them. This is just par for the course for Facebook, as a business they are so tone death and oblivious to basic human behavior and feelings that it seems like the entire company is infected with the same alleged autism of it's founder.

kelnos 9y ago
A) I think we're talking about modification-resistant fingerprinting methods, not just a simple hash of the data. B) That's a support nightmare; it would be nearly trivial to start submitting false reports from a ton of accounts without establishing a pattern so those accounts could easily be ignored. Regardless, how do you prove that you "own an image" (what does that even mean?) and that it was reported as an "intimate photo" maliciously? I agree that FB's current solution is really bad, and if it were me, I certainly wouldn't pro-actively send in photos of myself, but I'm at a loss for how they can address A & B.
- pavel_lishin 9y ago
  > That's a support nightmare; it would be nearly trivial to start submitting false reports from a ton of accounts without establishing a pattern so those accounts could easily be ignored. What's stopping me from flooding false reports now? Arguably, only reviewing the image they already have once a hash matches is less work, since they wouldn't have to review all incoming reports, only matched ones.
  - alsetmusic 9y ago
    > Arguably, only reviewing the image they already have once a hash matches is less work, since they wouldn't have to review all incoming reports, only matched ones. They would still need to investigate reports that didn’t match known hashes, but you’re right that this would be beneficial. Images that match a known hash could be prioritized during review.
    - pavel_lishin 9y ago
      Don't they have to do that now anyway?
- phkahler 9y ago
  The solution is to hash it locally and send the hash to Facebook without saving a local copy. Simple as that. Sure, someone could still do a lot of work to intercept or reverse engineer it and make an app to circumvent the hashing. But even if such an app existed, it would be obscure and people trying to upload unauthorized pics wouldn't bother or even know it exists. If you've got pics you don't want on the internet, the first rule is to not upload them to anywhere on the internet. Actually the real first rule is not to take them. Flash people your bits in person instead.
  - icebraining 9y ago
    Actually the real first rule is not to take them. Flash people your bits in person instead. Assuming you were the one to take the pictures, of course, and not got them from your ex-partner that's trying to pressure you by threatening to publish them on FB.
yawaramin 9y ago
Hey, please don't use phrasing like '... infected with ... autism'. First of all, autism isn't an infection, and second it dehumanises people with autism when you associate them with negative connotations like not being aware of human feelings and 'infecting' others. They are people too, don't dump on them.
- gruez 9y ago
  >First of all, autism isn't an infection it's being used as a rhetorical device. the implication here is that although autism isn't actually being spread per se, the environment at facebook (caused by profit motive or otherwise) causes "tone death and oblivious to basic human behavior and feelings". >second it dehumanises people with autism when you associate them with negative connotations like not being aware of human feelings im aware that autism exists on a spectrum, and there are high functioning autists, but isn't that at generally true?
  - yawaramin 9y ago
    For some people on the spectrum, yes, but you have to realise it's a severely frustrating experience for them as well because they have feelings and want to make human connections too. So it's not helpful to paint them as being oblivious. As I said, it's dehumanising. They deserve dignity and that includes being aware of the language we use about them.
  - scott_s 9y ago
    That it's being used as a rhetorical device is immaterial. It's still dehumanizing. And no, people on the autistic spectrum are fully aware of human feelings. But people on the autistic spectrum typically have difficulty reading other people's emotions.
  - duskwuff 9y ago
    > it's being used as a rhetorical device And it's an ugly, demeaning rhetorical device which insults a large group of people, some of whom are users of this site. (Not that it'd be any more acceptable if they weren't present.) Find another way of expressing yourself.
pvg 9y ago
It's definitely a hard problem, especially if you think about it a little rather than just dismissing it out of hand. For one thing, the hashes being discussed are resilient to some modifications - they're not general purpose hashes. More importantly, this is a case in which FB is making a serious attempt to deal with a complex issue rather than simply throwing up their hands and trotting out one of their standard excuses - 'it's not our fault, there isn't anything we can do, it's a side effect of the algorithm, we aren't responsible for how people use the platform' etc. This is a welcome change.
recursive 9y ago
> It's only going to catch those dumb enough not to modify the images You seem to be assuming they're using a general purpose hash, rather than one suitable for this purpose. That seems unlikely.

topmonk 9y ago

First, Facebook isn't the only place to share revenge porn, so they aren't really solving the problem, only reducing it slightly. Second, security by obscurity is not a good thing. Not running the algorithm locally, not to mention producing the source to their algorithm is not going to help in the long run. Third, regardless of whether the source of the algorithm is known or not, it would be a simple task to upload a test image of oneself, and then attempt to circumvent the algorithm using dummy accounts. Fourth, if someone abuses the service by uploading the hashes of a benign images, it can be reported at that time. No need to pre-emptively guard against this.

Havoc 9y ago

>Humans need to review uhm...this seems like a bit of a flaw in the plan

amptorn 9y ago

> Humans need to review to prevent adversarial reporting Humans already need to review the takedown reports themselves to prevent adversarial reporting!