24 ms·

Last paragraph of the article: "The new authentication scheme is the second in recent weeks that relies on photos. Earlier this month, Facebook asked users to

Last paragraph of the article: "The new authentication scheme is the second in recent weeks that relies on photos. Earlier this month, Facebook asked users to upload nude photos to Facebook Messenger, as part of an effort to prevent revenge porn. Facebook said it would use the nude photos to create a digital fingerprint against which to compare future posts." Wait what? I had to check whether today was April 1st.

jsonne 9y ago
I couldn't believe it. This is the most absurd thing I have ever heard.
- colordrops 9y ago
  It really does seem like something an elite private club would do to get a laugh out of fooling the proles while smoking a cigar and browsing the pictures.
  - Crespyl 9y ago
    > they "trust me" ... dumb f*cks
  - dwaltrip 9y ago
    It's more like oblivious high-paid product managers and devs who can't fathom why people wouldn't trust Facebook or that Facebook might not be a purely beneficial organization.
    - threeseed 9y ago
      I think you've completely missed the point here. Teenagers who have naked photos in the possession of their peers need a solution for preventing dissemination. And so if you're in this situation trusting Facebook is by far the lesser of two evils.
      - CamperBob2 9y ago
        How is Facebook legally able to solicit child pornography? Because it sounds like that's what they're doing.
- threeseed 9y ago
  Sexting is quite common about young adults and so is the subsequent sharing of those photos. Being able to stop that from happening is incredibly valuable. I am just surprised Facebook didn't instead look at a way of running these image hashing algorithms on the device instead.
  - madenine 9y ago
    I think they look for modifications of the photo as well so a cropping/tilt/text overlay/etc would still be flagged. Their model might require per-processing beyond what they can or want to deploy on-device.
  - golangnews 9y ago
    There is a much better way to stop it happening.
    - threeseed 9y ago
      And that would be what, exactly ?
austinprete 9y ago
Not sure how I feel about it still, but this article doesn't really accurately portray the service. From the source linked by the Wired article [1], this is specifically intended to prevent a nude photo from being shared maliciously when a user has reason to believe that it will be. They then voluntarily upload the image to Facebook which uses the digital fingerprint to prevent it from being reposted either in its original form or with slight modifications. It's always completely voluntary, Facebook is certainly not requiring or even really encouraging average users to upload their nudes. [1] - https://www.theverge.com/2017/11/9/16630900/facebook-revenge-porn-defense-details https://www.theverge.com/2017/11/9/16630900/facebook-revenge...
- cholantesh 9y ago
  I don't see how that makes it better.
  - cracell 9y ago
    Well in that case it actually makes sense. They can fingerprint the photo and then automatically remove it. Though really would be better to fingerprint all removed photos and automatically prevent any matching photos from being uploaded.
  - cbhl 9y ago
    Presumably, you would do this after Facebook already has a copy of the nude -- uploaded by the attacker/malicious ex. If you want Facebook's computers to automatically take down any copies of the photo, yes, you're going to have to trust them with a copy of it. Automated DMCA takedowns of copyrighted music and movies work in very much the same way.
    - the8472 9y ago
      > yes, you're going to have to trust them with a copy of it. No. They could have you compute a fingerprint locally without uploading the image itself, preferably through some audited open source software, without auto-updates. They only don't do that because they want to guard their image hashing as corporate secrets.
martin_marques 9y ago
We have "día de los inocentes" here, but it ain't that day either!
danso 9y ago
There was a thread about this on HN -- too lazy to look it up now or repeat some of the longer comments about it. But going into this assuming that FB has no ill-intentions, FB's proposal seems by far the best solution in a world of ugly and terrible solutions. For starters, it's intended for victims of revenge porn, which is a fairly extreme category and one in which the harassment is distinctively aggressive and virulent. Because if it weren't, the way FB deals with abusive content generally would seem to be good enough. If you come into this thinking that FB is asking everyone to upload their nudes for "safekeeping", then you've missed the point. Secondly, it's hard to think of an implementation that wouldn't create a potential disaster that justifies doing anything special for revenge porn victims. Using the Facebook app is the most secure channel for sending FB the photos because it is a secure app that all FB users know how to use. Having the user hash on their own requires either an external app or website. I don't think it needs to be said how such ancillary applications can be spoofed. Even if only 0.5% of users are dumb enough to fall for these spoofs, each incident would be a complete fucking disaster, for the victims and for Facebook. As for the prospect of FB owning people's nudes. Again, in the case of revenge porn victims, the horse is already far from the barn. If we assign the worst of motives to FB, that it's a way to secretly collect nudes from users. Again, horse, barn. This secret process would be less efficient by magnitudes compared to what FB can already do today.
- cracell 9y ago
  FB has a no porn policy so wtf do you have verify you are you to get a revenge porn picture removed? "We remove photographs of people displaying genitals or focusing in on fully exposed buttocks. We also restrict some images of female breasts if they include the nipple, but our intent is to allow images that are shared for medical or health purposes. We also allow photos of women actively engaged in breastfeeding or showing breasts with post-mastectomy scarring. We also allow photographs of paintings, sculptures, and other art that depicts nude figures. Restrictions on the display of sexual activity also apply to digitally created content unless the content is posted for educational, humorous, or satirical purposes. Explicit images of sexual intercourse are prohibited. Descriptions of sexual acts that go into vivid detail may also be removed." https://www.facebook.com/communitystandards#nudity https://www.facebook.com/communitystandards#nudity
bogomipz 9y ago
Same, I literally scrolled back to the top to make sure it wasn't The Onion. How was this not bigger news?
kbenson 9y ago
While this totally blows my mind, I can see how those a decade younger might not bat an eye, they might have sent digital nudes in the past, or even have one on the phone. A decade younger than that (late teens), and they might be so impatient about having to upload a digital nude, they just wonder why you can't just use one of the ones they sent last week. The difference in how the generations view technology and digital exposure in vast. "Sexting" came into the public consciousness as a common term almost a decade ago, but that's just when it was publicly popularized. I mean, I consciously think about every time I actually give a close approximation on my age in a public forum (like I did above), and make a call on whether it's needed or not. I feel like a dinosaur because of it sometimes, but I also feel like there's only so much privacy you're allotted, and once it's lost, there's really no getting it back. :/
i_am_nomad 9y ago
In context of Zuckerberg’s “They trust me, dumb fucks” comments, the invitation to upload nude photos seems especially alarming.
- Barrin92 9y ago
  misrepresents the idea of the service. It's supposed to tackle the 'revenge porn' issue. If someone sees their own nudes show up on facebook, they can send a picture to facebook, which I assume get's fed into an image detection service that then tries to remove the already circulating nudes automatically.
  - ProAm 9y ago
    There are way better and safer and less invasive ways to provide this service than uploading a personal photo to a 3rd party website.
    - abiox 9y ago
      > There are way better and safer and less invasive ways in theory or in practice? do these ways have other caveats?
      - musage 9y ago
        The only safe way is to not take nude pictures in the first place. What does this FB thing protect against? In practice? Upload the images as a zip file, message "hi, here's X in the nude, remove all fruit from htappletp://linkbananashortener.copearm/84395708345643785 and use "theytrustmedumbzuckerbergfucks" minus the FB founder as password". Yeah, it's a hurdle, but those who want to see the images will take it in a heartbeat, and abusers can still do things like message all mutual "friends", right there on FB. So you gained nothing, and pre-emptively gave FB your nude photos. Besides, Barrin92 said "If someone sees their own nudes show up on facebook" -- well, nude images aren't allowed on FB anyway, are they? So if you or anyone else can see them, if they're not posted in a private group or private messages, you can just report them, no need to even give away that it's you in them. Then what would stop FB from generating perceptual hashes of these images and removing them site-wide and in the future? There's so many holes in this that I can't really get mad on behalf of the people who fall for it.
dguest 9y ago
Can someone explain if / how this prevents someone from just changing a single pixel and thus circumventing the hash? Are they using some kind of probabilistic hashing? Or are they just relying on the offenders to not be so clever?
- the8472 9y ago
  There are perceptual image hashes[0]. There even are databases that have "search by similar image" indexing[1] [0] https://www.phash.org/ https://www.phash.org/ [1] http://www.lire-project.net/ http://www.lire-project.net/
- visarga 9y ago
  Neural nets, more exactly, convolutional neural nets. They are quite effective nowadays. Example: SphereFace: Deep Hypersphere Embedding for Face Recognition https://arxiv.org/abs/1704.08063v2 https://arxiv.org/abs/1704.08063v2
- threeseed 9y ago
  To be fair the type of people who are sharing naked photos via Facebook aren't likely to be all that technologically sophisticated to enough understand hashing. Pedophiles would be using something encrypted and anonymous anyway.
  - 659087 9y ago
    > Pedophiles would be using something encrypted and anonymous anyway. Except for the ones who apply for a job reviewing the nudes Facebook is collecting, anyway.
jacquesm 9y ago
That's so ridiculous it could have been written by The Onion. Facebook has absolutely no businesses to store nude pictures of their users.
- dymk 9y ago
  They don't. They store a perceptual hash, which can't be converted back into the original photo.
  - jacquesm 9y ago
    So you hope. But you have to upload your picture first. If they were serious about this they would allow you to compute the hash locally and then only to upload the hash. Note that they explicitly state that a human will look at the image and then hash it. So they are storing it at least for a while. And you will not be able to verify whether they really delete it or not.
    - abiox 9y ago
      > allow you to compute the hash locally presumably an attacker will just use the local process to observe and develop a method to defeat it. that said, perhaps such a methodology would require extra effort on the part of a revenge porn perp, so it might reduce the occurrences.
      - jacquesm 9y ago
        > presumably an attacker will just use the local process to observe and develop a method to defeat it. If they can do that then the whole method fails anyway. Besides, I think the set of your average revenge porn idiots intersected with those that are capable of defeating the hashing scheme in order to do their dirty deed is going to be exceedingly small.
        abiox 9y ago
        > If they can do that then the whole method fails anyway. i'm not that sure. the memory of any application can be observed, which means the process of fingerprinting - whatever it is - can be observed. executing the process enough times with a range of inputs will provide a load of analyzable data, via recording the transformation of bytes. this is (loosely) why basically all software can be "cracked" and drm methods defeated. > intersected with those that are capable of defeating the hashing scheme they don't really have to, much. think of the attackers as software crackers/warez groups, and the perps as visitors to pirate bay. you don't need to know how to break drm to download cracked software, or use nefarious push-button software. there might even be (possibly illicit) business opportunities here.
visarga 9y ago
Well played, sir.
coolaliasbro 9y ago
This is ridiculous. I mean, so many angles... Can't they just train their system to do whatever it is they propose that it do in response to alleged revenge porn, but with any nude image/nude images generally? Will their system prevent the propagation of revenge porn videos?
modzu 9y ago
loool
ljk 9y ago
basically it's "don't worry, people will review the image, but they're 'specially trained' " when will people stop using facebook?

jsonne 9y ago

I couldn't believe it. This is the most absurd thing I have ever heard.

colordrops 9y ago
It really does seem like something an elite private club would do to get a laugh out of fooling the proles while smoking a cigar and browsing the pictures.
- Crespyl 9y ago
  > they "trust me" ... dumb f*cks
- dwaltrip 9y ago
  It's more like oblivious high-paid product managers and devs who can't fathom why people wouldn't trust Facebook or that Facebook might not be a purely beneficial organization.
  - threeseed 9y ago
    I think you've completely missed the point here. Teenagers who have naked photos in the possession of their peers need a solution for preventing dissemination. And so if you're in this situation trusting Facebook is by far the lesser of two evils.
    - CamperBob2 9y ago
      How is Facebook legally able to solicit child pornography? Because it sounds like that's what they're doing.
threeseed 9y ago
Sexting is quite common about young adults and so is the subsequent sharing of those photos. Being able to stop that from happening is incredibly valuable. I am just surprised Facebook didn't instead look at a way of running these image hashing algorithms on the device instead.
- madenine 9y ago
  I think they look for modifications of the photo as well so a cropping/tilt/text overlay/etc would still be flagged. Their model might require per-processing beyond what they can or want to deploy on-device.
- golangnews 9y ago
  There is a much better way to stop it happening.
  - threeseed 9y ago
    And that would be what, exactly ?

austinprete 9y ago

Not sure how I feel about it still, but this article doesn't really accurately portray the service. From the source linked by the Wired article [1], this is specifically intended to prevent a nude photo from being shared maliciously when a user has reason to believe that it will be. They then voluntarily upload the image to Facebook which uses the digital fingerprint to prevent it from being reposted either in its original form or with slight modifications. It's always completely voluntary, Facebook is certainly not requiring or even really encouraging average users to upload their nudes. [1] - https://www.theverge.com/2017/11/9/16630900/facebook-revenge-porn-defense-details https://www.theverge.com/2017/11/9/16630900/facebook-revenge...

cholantesh 9y ago
I don't see how that makes it better.
- cracell 9y ago
  Well in that case it actually makes sense. They can fingerprint the photo and then automatically remove it. Though really would be better to fingerprint all removed photos and automatically prevent any matching photos from being uploaded.
- cbhl 9y ago
  Presumably, you would do this after Facebook already has a copy of the nude -- uploaded by the attacker/malicious ex. If you want Facebook's computers to automatically take down any copies of the photo, yes, you're going to have to trust them with a copy of it. Automated DMCA takedowns of copyrighted music and movies work in very much the same way.
  - the8472 9y ago
    > yes, you're going to have to trust them with a copy of it. No. They could have you compute a fingerprint locally without uploading the image itself, preferably through some audited open source software, without auto-updates. They only don't do that because they want to guard their image hashing as corporate secrets.

martin_marques 9y ago

We have "día de los inocentes" here, but it ain't that day either!

danso 9y ago

There was a thread about this on HN -- too lazy to look it up now or repeat some of the longer comments about it. But going into this assuming that FB has no ill-intentions, FB's proposal seems by far the best solution in a world of ugly and terrible solutions. For starters, it's intended for victims of revenge porn, which is a fairly extreme category and one in which the harassment is distinctively aggressive and virulent. Because if it weren't, the way FB deals with abusive content generally would seem to be good enough. If you come into this thinking that FB is asking everyone to upload their nudes for "safekeeping", then you've missed the point. Secondly, it's hard to think of an implementation that wouldn't create a potential disaster that justifies doing anything special for revenge porn victims. Using the Facebook app is the most secure channel for sending FB the photos because it is a secure app that all FB users know how to use. Having the user hash on their own requires either an external app or website. I don't think it needs to be said how such ancillary applications can be spoofed. Even if only 0.5% of users are dumb enough to fall for these spoofs, each incident would be a complete fucking disaster, for the victims and for Facebook. As for the prospect of FB owning people's nudes. Again, in the case of revenge porn victims, the horse is already far from the barn. If we assign the worst of motives to FB, that it's a way to secretly collect nudes from users. Again, horse, barn. This secret process would be less efficient by magnitudes compared to what FB can already do today.

cracell 9y ago
FB has a no porn policy so wtf do you have verify you are you to get a revenge porn picture removed? "We remove photographs of people displaying genitals or focusing in on fully exposed buttocks. We also restrict some images of female breasts if they include the nipple, but our intent is to allow images that are shared for medical or health purposes. We also allow photos of women actively engaged in breastfeeding or showing breasts with post-mastectomy scarring. We also allow photographs of paintings, sculptures, and other art that depicts nude figures. Restrictions on the display of sexual activity also apply to digitally created content unless the content is posted for educational, humorous, or satirical purposes. Explicit images of sexual intercourse are prohibited. Descriptions of sexual acts that go into vivid detail may also be removed." https://www.facebook.com/communitystandards#nudity https://www.facebook.com/communitystandards#nudity

bogomipz 9y ago

Same, I literally scrolled back to the top to make sure it wasn't The Onion. How was this not bigger news?

kbenson 9y ago

While this totally blows my mind, I can see how those a decade younger might not bat an eye, they might have sent digital nudes in the past, or even have one on the phone. A decade younger than that (late teens), and they might be so impatient about having to upload a digital nude, they just wonder why you can't just use one of the ones they sent last week. The difference in how the generations view technology and digital exposure in vast. "Sexting" came into the public consciousness as a common term almost a decade ago, but that's just when it was publicly popularized. I mean, I consciously think about every time I actually give a close approximation on my age in a public forum (like I did above), and make a call on whether it's needed or not. I feel like a dinosaur because of it sometimes, but I also feel like there's only so much privacy you're allotted, and once it's lost, there's really no getting it back. :/

i_am_nomad 9y ago

In context of Zuckerberg’s “They trust me, dumb fucks” comments, the invitation to upload nude photos seems especially alarming.

Barrin92 9y ago
misrepresents the idea of the service. It's supposed to tackle the 'revenge porn' issue. If someone sees their own nudes show up on facebook, they can send a picture to facebook, which I assume get's fed into an image detection service that then tries to remove the already circulating nudes automatically.
- ProAm 9y ago
  There are way better and safer and less invasive ways to provide this service than uploading a personal photo to a 3rd party website.
  - abiox 9y ago
    > There are way better and safer and less invasive ways in theory or in practice? do these ways have other caveats?
    - musage 9y ago
      The only safe way is to not take nude pictures in the first place. What does this FB thing protect against? In practice? Upload the images as a zip file, message "hi, here's X in the nude, remove all fruit from htappletp://linkbananashortener.copearm/84395708345643785 and use "theytrustmedumbzuckerbergfucks" minus the FB founder as password". Yeah, it's a hurdle, but those who want to see the images will take it in a heartbeat, and abusers can still do things like message all mutual "friends", right there on FB. So you gained nothing, and pre-emptively gave FB your nude photos. Besides, Barrin92 said "If someone sees their own nudes show up on facebook" -- well, nude images aren't allowed on FB anyway, are they? So if you or anyone else can see them, if they're not posted in a private group or private messages, you can just report them, no need to even give away that it's you in them. Then what would stop FB from generating perceptual hashes of these images and removing them site-wide and in the future? There's so many holes in this that I can't really get mad on behalf of the people who fall for it.

dguest 9y ago

Can someone explain if / how this prevents someone from just changing a single pixel and thus circumventing the hash? Are they using some kind of probabilistic hashing? Or are they just relying on the offenders to not be so clever?

the8472 9y ago
There are perceptual image hashes[0]. There even are databases that have "search by similar image" indexing[1] [0] https://www.phash.org/ https://www.phash.org/ [1] http://www.lire-project.net/ http://www.lire-project.net/
visarga 9y ago
Neural nets, more exactly, convolutional neural nets. They are quite effective nowadays. Example: SphereFace: Deep Hypersphere Embedding for Face Recognition https://arxiv.org/abs/1704.08063v2 https://arxiv.org/abs/1704.08063v2
threeseed 9y ago
To be fair the type of people who are sharing naked photos via Facebook aren't likely to be all that technologically sophisticated to enough understand hashing. Pedophiles would be using something encrypted and anonymous anyway.
- 659087 9y ago
  > Pedophiles would be using something encrypted and anonymous anyway. Except for the ones who apply for a job reviewing the nudes Facebook is collecting, anyway.

jacquesm 9y ago

That's so ridiculous it could have been written by The Onion. Facebook has absolutely no businesses to store nude pictures of their users.

dymk 9y ago
They don't. They store a perceptual hash, which can't be converted back into the original photo.
- jacquesm 9y ago
  So you hope. But you have to upload your picture first. If they were serious about this they would allow you to compute the hash locally and then only to upload the hash. Note that they explicitly state that a human will look at the image and then hash it. So they are storing it at least for a while. And you will not be able to verify whether they really delete it or not.
  - abiox 9y ago
    > allow you to compute the hash locally presumably an attacker will just use the local process to observe and develop a method to defeat it. that said, perhaps such a methodology would require extra effort on the part of a revenge porn perp, so it might reduce the occurrences.
    - jacquesm 9y ago
      > presumably an attacker will just use the local process to observe and develop a method to defeat it. If they can do that then the whole method fails anyway. Besides, I think the set of your average revenge porn idiots intersected with those that are capable of defeating the hashing scheme in order to do their dirty deed is going to be exceedingly small.
      - abiox 9y ago
        > If they can do that then the whole method fails anyway. i'm not that sure. the memory of any application can be observed, which means the process of fingerprinting - whatever it is - can be observed. executing the process enough times with a range of inputs will provide a load of analyzable data, via recording the transformation of bytes. this is (loosely) why basically all software can be "cracked" and drm methods defeated. > intersected with those that are capable of defeating the hashing scheme they don't really have to, much. think of the attackers as software crackers/warez groups, and the perps as visitors to pirate bay. you don't need to know how to break drm to download cracked software, or use nefarious push-button software. there might even be (possibly illicit) business opportunities here.

visarga 9y ago

Well played, sir.

coolaliasbro 9y ago

This is ridiculous. I mean, so many angles... Can't they just train their system to do whatever it is they propose that it do in response to alleged revenge porn, but with any nude image/nude images generally? Will their system prevent the propagation of revenge porn videos?

modzu 9y ago

loool

ljk 9y ago

basically it's "don't worry, people will review the image, but they're 'specially trained' " when will people stop using facebook?