4 ms·

I disagree: - It sets a precedent for uploading nude photos to FB and for them asking for it. - You need to trust FB to delete the photos when they receive it

I disagree: - It sets a precedent for uploading nude photos to FB and for them asking for it. - You need to trust FB to delete the photos when they receive it. Yes, I understand that they probably will, but really, how many systems are those bits going to touch? How many logs are going to have this information? Can you be really sure? A better implementation would be for the FB client to hash the file and for the hash to be uploaded. Trust issues are still there but at least mitigated to devices that you have some control over. Evil users that hash legitimate photos can be overcome with the same review system that is being tested today.

danso 9y ago
> - It sets a precedent for uploading nude photos to FB and for them asking for it. Before we reduce this to a slippery slope, what scenario do you envision in which FB could coax its userbase to upload nude photos? I know the OP is about taking a selfie to prove existence. What would FB use as the basis to mandate the general user to send a self-nude? > You need to trust FB to delete the photos when they receive it...can you really be sure? No, never, of course. But that's why I point out that FB already has this potential vector of attack. Every time a user flags content for abuse, that is logged and presumably a copy of the asset made for manual verification. Nevermind all the sensitive content millions of users everyday send across Messenger or private groups. > A better implementation would be for the FB client to hash the file and for the hash to be uploaded. If the image is hashed before it reaches FB servers, then it gives every user the power and impunity to attempt to censor via a Content-ID like approach.
- phjesusthatguy3 9y ago
  > Before we reduce this to a slippery slope, what scenario do you envision in which FB could coax its userbase to upload nude photos? The one where FB asks its userbase to upload nude photos[0]. [0]https://news.ycombinator.com/item?id=15651710 https://news.ycombinator.com/item?id=15651710
  - danso 9y ago
    That's exactly the topic we're discussing now. Sorry, with "userbase" I meant "general userbase". This initiative they're proposing -- in coordination with a safety group in Australia -- is aimed at revenge porn victims. The general userbase of Facebook aren't in that group.
    - phjesusthatguy3 9y ago
      I don't think there's any appropriate time for FB to request nude photos of their userbase. I don't understand why you think I should explain this.
      - Dylan16807 9y ago
        They aren't "requesting" it. The service exists for a specific reason, and is exceptionally optional.
- squeaky-clean 9y ago
  > If the image is hashed before it reaches FB servers, then it gives every user the power and impunity to attempt to censor via a Content-ID like approach. Another commenter makes a good point that you could still have a human verify the first time a provided hash matches an image. In the current setup, there is a human that verifies the image to be hashed is a nude photo instead of the McDonald's profile picture. In the proposed setup, you wait until a hash matches the photo and then have a human verify if it's pornographic content.
  - danso 9y ago
    The main obstacle that I can think of is: where does that hashing get done? Is it a feature that can efficiently be part of the phone app? Keeping in mind that this is a feature that would only be used by a very, very small part of the general userbase. Let's assume that it is possible, the other issue that might come up is that the system is still suspect to a sort of denial of service attack, in which a group (for whatever reason) floods FB with purported sensitive images, and FB is flooded with constant takedown requests to review.
    - Splines 9y ago
      The attack against the uploaded hash approach fails with default-deny, which seems like a good thing for users. The current implementation can be attacked by uploading thousands of legitimate images delaying takedown requests - therefore any images that should be taken down will stay up longer. I'll agree that whatever approach FB is doing to hash the photos may not work on a phone for technical reasons, but given FB's resources I'm not sure how far that argument really goes. But consider this - if you truly, deeply cared about user safety and privacy, would you implement this feature the same way?
      - danso 9y ago
        > The current implementation can be attacked by uploading thousands of legitimate images delaying takedown requests How would that work, exactly? A user uploads hundreds of fraudulent images to FB's revenge-porn-abuse queue. At some point, the human who verifies whether the image is legit is going to realize that the user account is fraudulent and then disable the account. If images are hashed, FB has no way to know if a user who is uploading hundreds of hashes is a malicious user or is actually an incredibly unfortunate revenge-porn victim. And the price for being wrong is extremely high. Maybe it's possible for FB's auto-detection system to be robust if the hashes it has to scan for is now several orders of magnitude than ever expected, making this all a moot point. But I can't imagine that the system scales with no penalty. > But consider this - if you truly, deeply cared about user safety and privacy, would you implement this feature the same way? The wording of your question implies a false dilemma, and I think reveals how different the premises you and I have about it. What exactly about Facebook's implementation of this feature makes it any less safe for users and their privacy than not having the feature at all? When a user sees and reports an abusive image of themselves -- that photo and that user, and that user's connection to the photo are already in Facebook's system". Every fear there is about this data being exposed to malicious human workers, or that FB is trying to harvest sensitive images for nefarious means -- that risk has always existed. How do you think abuse-takedown requests are currently handled? So if my argument is accurate, that an evil-pervy Facebook wants to do a mass collection of sensitive/comprising images of its user, all the infrastructure and dataflow is already in place, then this revenge-porn initiative does nothing to make that process more efficient. Even worse for pervy-Facebook, the initiative's very existence, nevermind announcing it, reminds the entire world again that holy-shit-think-of-all-the data-Facebook-has-on-us-including-our-sexy-times -- which is generally the kind of PR you want to avoid when you're conspiring to mass-harvest illicit imagery and data. And let's be real here: Facebook doesn't have to do anything special for revenge porn victims, in the way that the Postal Service isn't obligated to open everyone's mail to make absolutely sure there's no child porn being sent -- the act of prevention ends up causing far more harm to all users than it benefits the comparatively small number of potential victims. The status quo seems to be to do nothing until reports come in, which is OK for most situations but inadequate for the kind of attack vector that revenge-porn victims suffer. Facebook could have accepted that, as everyone else does, but invested time/resources into coming up with a technical solution that only benefits a very small but high-suffering part of its userbase while not increasing invasiveness (FB already autoscans the content of user messages, including with the use of PhotoDNA [0]). Call me Pollyannish, but I don't see this instance as yet another time of Facebook being heartless and devious. [0] http://www.businessinsider.com/facebook-google-and-microsofts-plan-to-ban-images-of-child-abuse-2013-7 http://www.businessinsider.com/facebook-google-and-microsoft...
- Splines 9y ago
  > Before we reduce this to a slippery slope, what scenario do you envision in which FB could coax its userbase to upload nude photos? I know the OP is about taking a selfie to prove existence. What would FB use as the basis to mandate the general user to send a self-nude? Phishing attacks against users that this feature is supposed to protect are more likely to succeed.
  - danso 9y ago
    Can you elaborate? People are worried (rightfully so) that this feature requires uploading via the Facebook Messenger App. But this means they don't have to visit a URL or download another application.
    - Splines 9y ago
      They don't necessarily know that. Imagine you've used this FB feature in the past. You get a mail from @facebookrnail.com that tells you you can just email them the photos without even opening the app now! Well isn't that convenient. Now, it's fair to say that even in a technically safer implementation where the photos never leave the device, many users can't tell the difference, so this point doesn't hold a lot of water. Still, I think making the uploading of scandalous photos to FB is a dangerous precedent to set in general. Will other services and startups that have users suffer from the same problem implement this feature in the same way, and guarantee user safety and privacy? That's a pretty high bar.
eh78ssxv2f 9y ago
> - It sets a precedent for uploading nude photos to FB and for them asking for it. umm, no. If I understand correctly, the user believes that their nude may end up on FB. The user takes initiative to upload their copy of the nude to FB to prove ownership or damages.
justadudeama 9y ago
Could a hash be tricked if the offender slightly modified the file, or even changed the format?

danso 9y ago

> - It sets a precedent for uploading nude photos to FB and for them asking for it. Before we reduce this to a slippery slope, what scenario do you envision in which FB could coax its userbase to upload nude photos? I know the OP is about taking a selfie to prove existence. What would FB use as the basis to mandate the general user to send a self-nude? > You need to trust FB to delete the photos when they receive it...can you really be sure? No, never, of course. But that's why I point out that FB already has this potential vector of attack. Every time a user flags content for abuse, that is logged and presumably a copy of the asset made for manual verification. Nevermind all the sensitive content millions of users everyday send across Messenger or private groups. > A better implementation would be for the FB client to hash the file and for the hash to be uploaded. If the image is hashed before it reaches FB servers, then it gives every user the power and impunity to attempt to censor via a Content-ID like approach.

phjesusthatguy3 9y ago
> Before we reduce this to a slippery slope, what scenario do you envision in which FB could coax its userbase to upload nude photos? The one where FB asks its userbase to upload nude photos[0]. [0]https://news.ycombinator.com/item?id=15651710 https://news.ycombinator.com/item?id=15651710
- danso 9y ago
  That's exactly the topic we're discussing now. Sorry, with "userbase" I meant "general userbase". This initiative they're proposing -- in coordination with a safety group in Australia -- is aimed at revenge porn victims. The general userbase of Facebook aren't in that group.
  - phjesusthatguy3 9y ago
    I don't think there's any appropriate time for FB to request nude photos of their userbase. I don't understand why you think I should explain this.
    - Dylan16807 9y ago
      They aren't "requesting" it. The service exists for a specific reason, and is exceptionally optional.
squeaky-clean 9y ago
> If the image is hashed before it reaches FB servers, then it gives every user the power and impunity to attempt to censor via a Content-ID like approach. Another commenter makes a good point that you could still have a human verify the first time a provided hash matches an image. In the current setup, there is a human that verifies the image to be hashed is a nude photo instead of the McDonald's profile picture. In the proposed setup, you wait until a hash matches the photo and then have a human verify if it's pornographic content.
- danso 9y ago
  The main obstacle that I can think of is: where does that hashing get done? Is it a feature that can efficiently be part of the phone app? Keeping in mind that this is a feature that would only be used by a very, very small part of the general userbase. Let's assume that it is possible, the other issue that might come up is that the system is still suspect to a sort of denial of service attack, in which a group (for whatever reason) floods FB with purported sensitive images, and FB is flooded with constant takedown requests to review.
  - Splines 9y ago
    The attack against the uploaded hash approach fails with default-deny, which seems like a good thing for users. The current implementation can be attacked by uploading thousands of legitimate images delaying takedown requests - therefore any images that should be taken down will stay up longer. I'll agree that whatever approach FB is doing to hash the photos may not work on a phone for technical reasons, but given FB's resources I'm not sure how far that argument really goes. But consider this - if you truly, deeply cared about user safety and privacy, would you implement this feature the same way?
    - danso 9y ago
      > The current implementation can be attacked by uploading thousands of legitimate images delaying takedown requests How would that work, exactly? A user uploads hundreds of fraudulent images to FB's revenge-porn-abuse queue. At some point, the human who verifies whether the image is legit is going to realize that the user account is fraudulent and then disable the account. If images are hashed, FB has no way to know if a user who is uploading hundreds of hashes is a malicious user or is actually an incredibly unfortunate revenge-porn victim. And the price for being wrong is extremely high. Maybe it's possible for FB's auto-detection system to be robust if the hashes it has to scan for is now several orders of magnitude than ever expected, making this all a moot point. But I can't imagine that the system scales with no penalty. > But consider this - if you truly, deeply cared about user safety and privacy, would you implement this feature the same way? The wording of your question implies a false dilemma, and I think reveals how different the premises you and I have about it. What exactly about Facebook's implementation of this feature makes it any less safe for users and their privacy than not having the feature at all? When a user sees and reports an abusive image of themselves -- that photo and that user, and that user's connection to the photo are already in Facebook's system". Every fear there is about this data being exposed to malicious human workers, or that FB is trying to harvest sensitive images for nefarious means -- that risk has always existed. How do you think abuse-takedown requests are currently handled? So if my argument is accurate, that an evil-pervy Facebook wants to do a mass collection of sensitive/comprising images of its user, all the infrastructure and dataflow is already in place, then this revenge-porn initiative does nothing to make that process more efficient. Even worse for pervy-Facebook, the initiative's very existence, nevermind announcing it, reminds the entire world again that holy-shit-think-of-all-the data-Facebook-has-on-us-including-our-sexy-times -- which is generally the kind of PR you want to avoid when you're conspiring to mass-harvest illicit imagery and data. And let's be real here: Facebook doesn't have to do anything special for revenge porn victims, in the way that the Postal Service isn't obligated to open everyone's mail to make absolutely sure there's no child porn being sent -- the act of prevention ends up causing far more harm to all users than it benefits the comparatively small number of potential victims. The status quo seems to be to do nothing until reports come in, which is OK for most situations but inadequate for the kind of attack vector that revenge-porn victims suffer. Facebook could have accepted that, as everyone else does, but invested time/resources into coming up with a technical solution that only benefits a very small but high-suffering part of its userbase while not increasing invasiveness (FB already autoscans the content of user messages, including with the use of PhotoDNA [0]). Call me Pollyannish, but I don't see this instance as yet another time of Facebook being heartless and devious. [0] http://www.businessinsider.com/facebook-google-and-microsofts-plan-to-ban-images-of-child-abuse-2013-7 http://www.businessinsider.com/facebook-google-and-microsoft...
Splines 9y ago
> Before we reduce this to a slippery slope, what scenario do you envision in which FB could coax its userbase to upload nude photos? I know the OP is about taking a selfie to prove existence. What would FB use as the basis to mandate the general user to send a self-nude? Phishing attacks against users that this feature is supposed to protect are more likely to succeed.
- danso 9y ago
  Can you elaborate? People are worried (rightfully so) that this feature requires uploading via the Facebook Messenger App. But this means they don't have to visit a URL or download another application.
  - Splines 9y ago
    They don't necessarily know that. Imagine you've used this FB feature in the past. You get a mail from @facebookrnail.com that tells you you can just email them the photos without even opening the app now! Well isn't that convenient. Now, it's fair to say that even in a technically safer implementation where the photos never leave the device, many users can't tell the difference, so this point doesn't hold a lot of water. Still, I think making the uploading of scandalous photos to FB is a dangerous precedent to set in general. Will other services and startups that have users suffer from the same problem implement this feature in the same way, and guarantee user safety and privacy? That's a pretty high bar.

eh78ssxv2f 9y ago

> - It sets a precedent for uploading nude photos to FB and for them asking for it. umm, no. If I understand correctly, the user believes that their nude may end up on FB. The user takes initiative to upload their copy of the nude to FB to prove ownership or damages.

justadudeama 9y ago

Could a hash be tricked if the offender slightly modified the file, or even changed the format?