3 ms·

The attack against the uploaded hash approach fails with default-deny, which seems like a good thing for users. The current implementation can be attacked by u

The attack against the uploaded hash approach fails with default-deny, which seems like a good thing for users. The current implementation can be attacked by uploading thousands of legitimate images delaying takedown requests - therefore any images that should be taken down will stay up longer. I'll agree that whatever approach FB is doing to hash the photos may not work on a phone for technical reasons, but given FB's resources I'm not sure how far that argument really goes. But consider this - if you truly, deeply cared about user safety and privacy, would you implement this feature the same way?

danso 9y ago
> The current implementation can be attacked by uploading thousands of legitimate images delaying takedown requests How would that work, exactly? A user uploads hundreds of fraudulent images to FB's revenge-porn-abuse queue. At some point, the human who verifies whether the image is legit is going to realize that the user account is fraudulent and then disable the account. If images are hashed, FB has no way to know if a user who is uploading hundreds of hashes is a malicious user or is actually an incredibly unfortunate revenge-porn victim. And the price for being wrong is extremely high. Maybe it's possible for FB's auto-detection system to be robust if the hashes it has to scan for is now several orders of magnitude than ever expected, making this all a moot point. But I can't imagine that the system scales with no penalty. > But consider this - if you truly, deeply cared about user safety and privacy, would you implement this feature the same way? The wording of your question implies a false dilemma, and I think reveals how different the premises you and I have about it. What exactly about Facebook's implementation of this feature makes it any less safe for users and their privacy than not having the feature at all? When a user sees and reports an abusive image of themselves -- that photo and that user, and that user's connection to the photo are already in Facebook's system". Every fear there is about this data being exposed to malicious human workers, or that FB is trying to harvest sensitive images for nefarious means -- that risk has always existed. How do you think abuse-takedown requests are currently handled? So if my argument is accurate, that an evil-pervy Facebook wants to do a mass collection of sensitive/comprising images of its user, all the infrastructure and dataflow is already in place, then this revenge-porn initiative does nothing to make that process more efficient. Even worse for pervy-Facebook, the initiative's very existence, nevermind announcing it, reminds the entire world again that holy-shit-think-of-all-the data-Facebook-has-on-us-including-our-sexy-times -- which is generally the kind of PR you want to avoid when you're conspiring to mass-harvest illicit imagery and data. And let's be real here: Facebook doesn't have to do anything special for revenge porn victims, in the way that the Postal Service isn't obligated to open everyone's mail to make absolutely sure there's no child porn being sent -- the act of prevention ends up causing far more harm to all users than it benefits the comparatively small number of potential victims. The status quo seems to be to do nothing until reports come in, which is OK for most situations but inadequate for the kind of attack vector that revenge-porn victims suffer. Facebook could have accepted that, as everyone else does, but invested time/resources into coming up with a technical solution that only benefits a very small but high-suffering part of its userbase while not increasing invasiveness (FB already autoscans the content of user messages, including with the use of PhotoDNA [0]). Call me Pollyannish, but I don't see this instance as yet another time of Facebook being heartless and devious. [0] http://www.businessinsider.com/facebook-google-and-microsofts-plan-to-ban-images-of-child-abuse-2013-7 http://www.businessinsider.com/facebook-google-and-microsoft...
- drewmol 9y ago
  ^This They could jump straight to using ml and ai and the future where only you are in charge of who sees your FacebookEroticExpressions™ on any platform, all across the cloud! But in the mean time they should probably tell her/him/them if they use the method that already exist to upload them we can use the method that already exist to have them gone in time for recess/study hall/wedding day/when Congress resumes/his state of the Union Address this evening/etc. Also, if you just give us the other ones that undoubtedly exist we can nip this whole thing in the bud right now.

danso 9y ago

> The current implementation can be attacked by uploading thousands of legitimate images delaying takedown requests How would that work, exactly? A user uploads hundreds of fraudulent images to FB's revenge-porn-abuse queue. At some point, the human who verifies whether the image is legit is going to realize that the user account is fraudulent and then disable the account. If images are hashed, FB has no way to know if a user who is uploading hundreds of hashes is a malicious user or is actually an incredibly unfortunate revenge-porn victim. And the price for being wrong is extremely high. Maybe it's possible for FB's auto-detection system to be robust if the hashes it has to scan for is now several orders of magnitude than ever expected, making this all a moot point. But I can't imagine that the system scales with no penalty. > But consider this - if you truly, deeply cared about user safety and privacy, would you implement this feature the same way? The wording of your question implies a false dilemma, and I think reveals how different the premises you and I have about it. What exactly about Facebook's implementation of this feature makes it any less safe for users and their privacy than not having the feature at all? When a user sees and reports an abusive image of themselves -- that photo and that user, and that user's connection to the photo are already in Facebook's system". Every fear there is about this data being exposed to malicious human workers, or that FB is trying to harvest sensitive images for nefarious means -- that risk has always existed. How do you think abuse-takedown requests are currently handled? So if my argument is accurate, that an evil-pervy Facebook wants to do a mass collection of sensitive/comprising images of its user, all the infrastructure and dataflow is already in place, then this revenge-porn initiative does nothing to make that process more efficient. Even worse for pervy-Facebook, the initiative's very existence, nevermind announcing it, reminds the entire world again that holy-shit-think-of-all-the data-Facebook-has-on-us-including-our-sexy-times -- which is generally the kind of PR you want to avoid when you're conspiring to mass-harvest illicit imagery and data. And let's be real here: Facebook doesn't have to do anything special for revenge porn victims, in the way that the Postal Service isn't obligated to open everyone's mail to make absolutely sure there's no child porn being sent -- the act of prevention ends up causing far more harm to all users than it benefits the comparatively small number of potential victims. The status quo seems to be to do nothing until reports come in, which is OK for most situations but inadequate for the kind of attack vector that revenge-porn victims suffer. Facebook could have accepted that, as everyone else does, but invested time/resources into coming up with a technical solution that only benefits a very small but high-suffering part of its userbase while not increasing invasiveness (FB already autoscans the content of user messages, including with the use of PhotoDNA [0]). Call me Pollyannish, but I don't see this instance as yet another time of Facebook being heartless and devious. [0] http://www.businessinsider.com/facebook-google-and-microsofts-plan-to-ban-images-of-child-abuse-2013-7 http://www.businessinsider.com/facebook-google-and-microsoft...

drewmol 9y ago
^This They could jump straight to using ml and ai and the future where only you are in charge of who sees your FacebookEroticExpressions™ on any platform, all across the cloud! But in the mean time they should probably tell her/him/them if they use the method that already exist to upload them we can use the method that already exist to have them gone in time for recess/study hall/wedding day/when Congress resumes/his state of the Union Address this evening/etc. Also, if you just give us the other ones that undoubtedly exist we can nip this whole thing in the bud right now.