4 ms·

From the fine article: "We store the photo hash—not the photo—to prevent someone from uploading the photo in the future. If someone tries to upload the image t

by moyix 9y ago

From the fine article: "We store the photo hash—not the photo—to prevent someone from uploading the photo in the future. If someone tries to upload the image to our platform, like all photos on Facebook, it is run through a database of these hashes and if it matches we do not allow it to be posted or shared."

BadassFractal 9y ago
Ah, missed that, thanks. Yeah I guess you have to trust that part. I guess I'm willing to believe that. Hopefully it's not like with Snapchat where everything get stored behind the scenes forever even though it "disappears". Also, wonder if basing things on hash will make detecting slightly modified versions of that content almost impossible. E.g. IG / YT can detect if you upload content with a piece of music that's copyrighted regardless of how much you alter it, but this would be fairly primitive.
tonyarkles 9y ago
Is there a way for me to audit that the image is never stored? (No) It seems like that's a huge honeypot; someone compromises the "upload your naughty photos here" endpoint and has an endless supply of "non-consensual intimate images". Edit: Thinking about it a bit, I'd be way more comfortable with the idea of the image hashes being computed on-device and only the hashes being sent to the server. This opens a different door of abuse (by essentially permitting individuals to ban someone from posting an image by uploading the hash of the image), but it's definitely preferable to encouraging people to send all their selfies to the privacy commissioner.
- evgen 9y ago
  While this (client-side hashing) would be better in terms of privacy protection for this specific case it is not going to happen because Facebook is not allowed to perform the specific photo hashing client-side as this would expose the hashing mechanism to analysis.
  - dmitrygr 9y ago
    > this would expose the hashing mechanism to analysis Security through obscurity. Works every time (tm)
    - kelnos 9y ago
      I mean, it worked pretty well for Google's ranking algorithm, so much so that it spawned an entire industry (SEO, which still doesn't let you get away with much in the way of black-hat type stuff). Even just the rate-limiting aspect of keeping the algo on the server (in this case behind a human, but that's even not necessary) is useful: I can't sit offline with the algo and keep beating it with slightly different images until the hash changes enough to be considered unique.
  - the8472 9y ago
    maybe they shouldn't ask for nudes until they have designed an open source algorithm?
    - 15155 9y ago
      How would an open source fingerprinting algorithm work in this case? There would be a browser extension to "refingerprint" images within 24 hours of release.
      - the8472 9y ago
        Make the algorithms public, let them be broken, refine them etc. until the margin between identification and refingerprinting becomes so small that refingerprinting involves such significant changes that we are reasonably confident that we are no longer dealing with theseus' ship. That's how we got good crypto. By not keeping it proprietary. And a side benefit that we would get some amazingly alteration-resistant perceptual image hashes out of that.
        Freak_NL 9y ago
        It would be interesting to see what people come up with, but I doubt any algorithm can stop someone dedicated to figure out a way to defeat it. Having the algorithm open for public review will lead to better algorithms, but allow for direct feedback if you are attempting to circumvent it; whilst keeping it proprietary will prevent direct feedback, but might keep the algorithm less clever than it could be. If Facebook is using just PhotoDNA, defeating it should be trivial¹: > […] It works by converting the image to black and white, re-sizing it, breaking it into a grid, and looking at intensity gradients or edges […] That's fairly generic, but effective enough for resizing, colour adjustments, minor changes, etc., but not good enough for porn and an adversary willing to do some experimenting with the photo. If I were to guess at how Facebook uses this, I would expect that they use a preprocessing routine that strips out anything that doesn't resemble a nude or semi-nude person, to prevent alterations to the person's surroundings from changing the hash — which is what you would do if you wanted to bypass the filter whilst keeping the essence of the picture intact. Keeping that fact hidden would be integral to their strategy I suppose. 1: https://en.wikipedia.org/wiki/PhotoDNA https://en.wikipedia.org/wiki/PhotoDNA
BinaryIdiot 9y ago
But you are sending an image to yourself in Messenger which DOES normally keep an image. It doesn't say here that they explicitly delete the message you sent via Messenger, does it? Perhaps you have to delete that for yourself? And do they take back-ups of Messenger data in which this photo may now be in? Seems like a very awkward flow to forcefully repurpose Messenger to do something it shouldn't be doing.
giobox 9y ago
All you can do is take FB at their word that the image is destroyed irrevocably. Given what we know about data retention policies at big tech firms, I'm not so sure I would feel confident taking the press release at face value. I'd really like to see a white paper or similar outlining the specifics of how this is being handled. I'd also have questions around what steps have been taken to prevent rogue Facebook employees trying to obtain the images. I really wish they had found a way to generate the hashes they require client side and not receive the image at all, especially as this is something that presumably would be really great for revenge porn victims.
- beager 9y ago
  I don’t know. I think a whitepaper might just be a drier way to lie about what they’re really doing.
retrogradeorbit 9y ago
You have to read this as a lawyer would. Let me show you: We store the photo hash - not the photo. This says nothing about whether other parties we pass the data onto or through store the photo. Remember when prism was leaked and some tech companies said 'we do not give our data to the nsa'. give. (Of course we provided them with a backdoor login.)

by moyix 9y ago

BadassFractal 9y ago

Ah, missed that, thanks. Yeah I guess you have to trust that part. I guess I'm willing to believe that. Hopefully it's not like with Snapchat where everything get stored behind the scenes forever even though it "disappears". Also, wonder if basing things on hash will make detecting slightly modified versions of that content almost impossible. E.g. IG / YT can detect if you upload content with a piece of music that's copyrighted regardless of how much you alter it, but this would be fairly primitive.

tonyarkles 9y ago

Is there a way for me to audit that the image is never stored? (No) It seems like that's a huge honeypot; someone compromises the "upload your naughty photos here" endpoint and has an endless supply of "non-consensual intimate images". Edit: Thinking about it a bit, I'd be way more comfortable with the idea of the image hashes being computed on-device and only the hashes being sent to the server. This opens a different door of abuse (by essentially permitting individuals to ban someone from posting an image by uploading the hash of the image), but it's definitely preferable to encouraging people to send all their selfies to the privacy commissioner.

evgen 9y ago
While this (client-side hashing) would be better in terms of privacy protection for this specific case it is not going to happen because Facebook is not allowed to perform the specific photo hashing client-side as this would expose the hashing mechanism to analysis.
- dmitrygr 9y ago
  > this would expose the hashing mechanism to analysis Security through obscurity. Works every time (tm)
  - kelnos 9y ago
    I mean, it worked pretty well for Google's ranking algorithm, so much so that it spawned an entire industry (SEO, which still doesn't let you get away with much in the way of black-hat type stuff). Even just the rate-limiting aspect of keeping the algo on the server (in this case behind a human, but that's even not necessary) is useful: I can't sit offline with the algo and keep beating it with slightly different images until the hash changes enough to be considered unique.
- the8472 9y ago
  maybe they shouldn't ask for nudes until they have designed an open source algorithm?
  - 15155 9y ago
    How would an open source fingerprinting algorithm work in this case? There would be a browser extension to "refingerprint" images within 24 hours of release.
    - the8472 9y ago
      Make the algorithms public, let them be broken, refine them etc. until the margin between identification and refingerprinting becomes so small that refingerprinting involves such significant changes that we are reasonably confident that we are no longer dealing with theseus' ship. That's how we got good crypto. By not keeping it proprietary. And a side benefit that we would get some amazingly alteration-resistant perceptual image hashes out of that.
      - Freak_NL 9y ago
        It would be interesting to see what people come up with, but I doubt any algorithm can stop someone dedicated to figure out a way to defeat it. Having the algorithm open for public review will lead to better algorithms, but allow for direct feedback if you are attempting to circumvent it; whilst keeping it proprietary will prevent direct feedback, but might keep the algorithm less clever than it could be. If Facebook is using just PhotoDNA, defeating it should be trivial¹: > […] It works by converting the image to black and white, re-sizing it, breaking it into a grid, and looking at intensity gradients or edges […] That's fairly generic, but effective enough for resizing, colour adjustments, minor changes, etc., but not good enough for porn and an adversary willing to do some experimenting with the photo. If I were to guess at how Facebook uses this, I would expect that they use a preprocessing routine that strips out anything that doesn't resemble a nude or semi-nude person, to prevent alterations to the person's surroundings from changing the hash — which is what you would do if you wanted to bypass the filter whilst keeping the essence of the picture intact. Keeping that fact hidden would be integral to their strategy I suppose. 1: https://en.wikipedia.org/wiki/PhotoDNA https://en.wikipedia.org/wiki/PhotoDNA

BinaryIdiot 9y ago

But you are sending an image to yourself in Messenger which DOES normally keep an image. It doesn't say here that they explicitly delete the message you sent via Messenger, does it? Perhaps you have to delete that for yourself? And do they take back-ups of Messenger data in which this photo may now be in? Seems like a very awkward flow to forcefully repurpose Messenger to do something it shouldn't be doing.

giobox 9y ago

All you can do is take FB at their word that the image is destroyed irrevocably. Given what we know about data retention policies at big tech firms, I'm not so sure I would feel confident taking the press release at face value. I'd really like to see a white paper or similar outlining the specifics of how this is being handled. I'd also have questions around what steps have been taken to prevent rogue Facebook employees trying to obtain the images. I really wish they had found a way to generate the hashes they require client side and not receive the image at all, especially as this is something that presumably would be really great for revenge porn victims.

beager 9y ago
I don’t know. I think a whitepaper might just be a drier way to lie about what they’re really doing.

retrogradeorbit 9y ago

You have to read this as a lawyer would. Let me show you: We store the photo hash - not the photo. This says nothing about whether other parties we pass the data onto or through store the photo. Remember when prism was leaked and some tech companies said 'we do not give our data to the nsa'. give. (Of course we provided them with a backdoor login.)