8 ms·

Google, Microsoft, and Facebook have been reported as having a list of image hashes which they very strongly do not want you sending through their systems. If

by patio11 12y ago

Google, Microsoft, and Facebook have been reported as having a list of image hashes which they very strongly do not want you sending through their systems. If you attempt to do so, that won't work out well for you. One writeup of many: http://www.microsoft.com/en-us/news/features/2009/dec09/12-15photodna.aspx http://www.microsoft.com/en-us/news/features/2009/dec09/12-1...

eloff 12y ago
That makes more sense, the article makes it sound like Google engineers were browsing through people's private photos, at least that's what every non-technical person who reads it will take away. It seems obvious to me that it must have been some kind of automated process that flagged the images.
- bowlofpetunias 12y ago
  A non-technical person will not consider that any different. After all, said automated process was created and deployed by Google engineers, so it's still Google engineers browsing through people's private photos. Just because I use a tool doesn't mean I'm no longer responsible for my actions. It's exactly this complete dissociation of responsibility for the consequences why we will be facing an increasing anti-techie backlash. It's time we stopped hiding behind semi-autistic semantics. Google engineers went through people's private photos. Period. What tools they used to do it is utterly irrelevant.
  - icebraining 12y ago
    A non-technical person will not consider that any different. That's not my experience at all. I stopped using Gmail in part due to privacy concerns, and when I explain it to non-techies, they're OK with it specifically because it's software and not people looking at the photos. People, in my experience, aren't worried about potential false positives or abuses of power. They just don't like the idea of other persons looking at their private pictures.
  - JoshTriplett 12y ago
    This is not 4chan; "autistic" is not an appropriate description for anything other than the actual psychological disorder.
    - tomjen3 12y ago
      This is the internet. If you are going to be offended over the use of words you are going to have a bad time and should log of right now.
      - corpselord420 12y ago
        this is a website with a community. if you're going to prioritize some asshole's right to be a bigot over being inclusive then you're welcome to fuck right off.
      - AndrewDucker 12y ago
        This is a site on the internet. Which doesn't mean that it can't have standards of conduct that make it possible to have civilised discussion.
        Pacabel 12y ago
        When read without putting political correctness above all else, the comment from bowlofpetunias is quite civilized. I think it brings up some very good points about how non-technical people and technical people can interpret situations involving technology in very different ways, for example. And I think it's actually trying to hold the technical community to a much higher standard when it comes to justifying actions that society at large will very likely disagree with. It's far more harmful and disruptive to see the perfectly legitimate argument of bowlofpetunias being totally ignored and even obscured, merely because some people find a single word in the comment to potentially be politically incorrect or potentially offensive. Discussion is much better when the focus is on the argument or topic itself, rather than going off on political correctness tangents over a single word in somebody's comment.
      - 27182818284 12y ago
  - DanBC 12y ago
    > It's time we stopped hiding behind semi-autistic semantics. Semi-autistic is pretty fucking offensive. > Google engineers went through people's private photos. Period. What tools they used to do it is utterly irrelevant. Ignoring for a moment the fact that the whole point of Google is to inspect your traffic in order to serve you ads (and that's been used by competitors in their own adverts before; see Microsoft) there's a big difference between taking[1] image hashes to then compare to hashes of data passing through your network and some person sat at a desk looking at the images. [1] Probably being given image hashes by law enforcement. A much more interesting question is how fast they can do this? Imagine a set of say 500,000 hashes of the worst images. And global Google traffic. That's a lot of hashing and comparing going on. And are they doing simplistic hash checking, or is it more complicated so that people can't just change a few pixels to evade the hashes?
    - morsch 12y ago
      Wait wait wait, so it's acceptable now that government/LEO give a list of forbidden hashes to service providers with the expectation that they filter their networks and report anybody who's a match? Because that sounds bat shit insane. Who's to say protest-plans.ppt won't be the next thing on the list? Or justin-bieber.mp3? Or cia-torture-prisons.txt? With a hash, the ISP doesn't even know what the file is about before they detect a match, and having them look closer at matches and decide whether the government needs to know is just as insane.
      - meowface 12y ago
        >Who's to say protest-plans.ppt won't be the next thing on the list? Or justin-bieber.mp3? Or cia-torture-prisons.txt? Because possession of those files doesn't break federal law, but possession of child pornography does? And even if those things were added, obviously Google does some sort of rough verification process of the nature of the content being sent if it matches a hash. Service providers have been sharing image hashes of CP for 6+ years now. It's definitely not just Google. Your slippery slope makes no sense. This is probably the absolute least invasive thing that Google does. You are aware they show ads based on the actual text content of your emails, right?
- Excavator 12y ago
  On that topic: http://articles.latimes.com/2010/sep/16/business/la-fi-google-firing-20100916 http://articles.latimes.com/2010/sep/16/business/la-fi-googl... http://gawker.com/5637234/gcreep-google-engineer-stalked-teens-spied-on-chats http://gawker.com/5637234/gcreep-google-engineer-stalked-tee...
- GeneralMayhem 12y ago
  That's also what any non-technical person who thinks about for more than a millisecond would realize too, given the volume of Gmail traffic. Unfortunately, a millisecond's thought isn't conducive to fearmongering.
  - dllthomas 12y ago
    A millisecond's thought will tell you that Google engineers aren't looking at every email (or a substantive fraction). That doesn't mean they aren't looking at any email, which is all that is necessary for that interpretation.
    - tptacek 12y ago
      The latter interpretation, if I understand it correctly, is that Google engineers randomly poke at people's emails and in this case just happened to get lucky. Even stipulating thousands of Google engineers doing that, that doesn't sound statistically plausible; the base rate of child pornography among Google Mail users is extremely low.
      - eloff 12y ago
        That was my thoughts as well. But I don't think most people think that way, especially given the misleading wording of the article. I hate to contribute to the "rest of the world is stupid" theory, or pretend that I'm immune from such things, but in this case I really do think the vast majority of people won't evaluate things in a probabilistic way like that. Most people just aren't trained to think in terms of odds instead of emotions (and it constantly annoys me.) At least that is my experience, and it may not be representative. Anecdote, I go swimming in a river known to have Caiman (small alligators) downstream and everyone thinks I'm a huge risk taker. Nobody comments on the 3 hour drive to get there and back in heavy rain and high speeds. It's obvious, mathematically speaking, which is the riskier activity.
      - dllthomas 12y ago
        I agree that it still seems unlikely, but it requires deeper thought to deduce that than "Google engineers can't possibly be looking at every email".
psykovsky 12y ago
PhotoDNA[0], for the ones interested in knowing a little bit more [0] http://en.wikipedia.org/wiki/PhotoDNA http://en.wikipedia.org/wiki/PhotoDNA
- onewaystreet 12y ago
  Anyone know of a similar technology (generating a hash from an image that is resistant to alterations) that is open source?
  - krapp 12y ago
    If you don't mind php, I wrote this ages ago specifically to catch image spam on forums... the project is more or less dead now because i'm not technically capable of improving it, but it does work (albeit with false positives in some instances): https://github.com/kennethrapp/phasher https://github.com/kennethrapp/phasher You may find the actual algorithm used more interesting (http://phash.org/ http://phash.org/)
  - troels 12y ago
    I'm fairly sure [OpenCV](http://opencv.org http://opencv.org) can do that for you.
dm2 12y ago
Here is a very simple overview of how it works: http://www.microsoft.com/global/en-us/news/publishingimages/ImageGallery/Images/Infographics/PhotoDNA/flowchart_photodna_Print.jpg http://www.microsoft.com/global/en-us/news/publishingimages/...
bcoates 12y ago
That's a nice, subtle way of fucking with someone: spoofing messages that trip these pedo-sensors won't result in an obvious mistaken raid, because the police will want something more solid to go on (like the case in the OP) -- the victim will just show up as 'pedophile, we just can't prove it yet' in a bunch of police databases/background checks/opposition research, and won't know why the random searches randomly choose them every time.
- tptacek 12y ago
  There's no such thing as a "pedo-sensor". This is a hash-based search against a corpus of known images, where the hash function was designed explicitly (a) to survive image transformations and (b) to prevent unauthorized parties from predicting the hash of a given image. Either of those two goals could fail and it would still be implausible that an attacker could trigger a false positive without using an image derived from actual known child pornography.
  - true_religion 12y ago
    I think he means spoofing the message was sent by a particular person, but not actually spoofing the content. The content has to be real to trip the hash-search.
    - deleted 12y ago
      [deleted]
  - dredmorbius 12y ago
    I'm actually aware of systems which were built to identify porn through neural-network training. I don't know if these were sensitive to distinguish child porn from other, but as a method of finding images likely to contain more skin than desired, they worked fairly reliably. That said, the Google/Microsoft tool does seem to work based on a known image corpus.
- joeframbach 12y ago
  Perhaps this may change if a certain list of 535 people in DC are added to this list.
doe88 12y ago
And what happens if someone (who wish you bad) anonymously send you such images as if he was your longtime buddy? Do they also report you?
- deleted 12y ago
  [deleted]
lifeisstillgood 12y ago
From the write up >> Since 2003, NCMEC has reviewed and analyzed almost 30 million images and videos of child pornography I deeply deeply hope that is not 30M distinct photos ... thats an industrial level output not a few ugly people trafficking in horror. I was struck by a recent podcast from BBCs More Or Less which tried to estimate number of paedophiles in population - and came up with 1%-5%. The problem was the definition - sexual thoughts, sexual actions with pre-pubescent of pubescent children upto the age of consent. That lumped people who thought Kate Moss' early nude photos were sexy together with people who rape 4 year olds which I don't think helps the debate. Anyway - I think the prevalence is greater than is generally recognised - but even so - 30 M images seems enormous.
dredmorbius 12y ago
The question of what Google might search for, using what hashes, on whom, why and when, and how, comes to mind. Google didn't "look at pictures". It had a database of known child porn images for which it computed hashes or signatures, which then could be checked, presumably of attachments in emails. This lets Google avoid the matter of "we looked at your specific images". It also means that your random personal photos aren't going to be examined (though their hashes are computed, and possibly retained). The hashing approach could apply to other aspects of email as well, though. Google could, perhaps, be encouraged to maintain a set of hashes corresponding to musical performances (it uses a similar technology AFAIU for YouTube copyright determination). Or of books. It could even take tuples of word sequences within email, compute hashes, and compare these with known word strings, forwarding relevant information to interested parties should certain sequences occur. All without "reading" your email. Where do you draw the line? Is it "think of the children"? Do we extend this to "think of the terrorists"? How about seeking out white-collar crime -- a set of account numbers, or key phrases, or transaction sequences? Or do we look for drugs crimes? Or gang activity? Or political insurrection? Can different national jurisdictions come up with their own sets of keywords? Say, those associated with specific religious or ethnic groups and activities? Uighurs, or Tibetan independence movement? Boko Haram? Aum Shinrikyo? Provisional IRA? Shining Path? ELF and PETA? Bangsamoro Islamic Freedom Fighters? Because those are all possible with slight variations on the methods described here. Or how should (or must) Google respond if users employ encryption to defeat such attempted detection?

by patio11 12y ago

eloff 12y ago

That makes more sense, the article makes it sound like Google engineers were browsing through people's private photos, at least that's what every non-technical person who reads it will take away. It seems obvious to me that it must have been some kind of automated process that flagged the images.

bowlofpetunias 12y ago
A non-technical person will not consider that any different. After all, said automated process was created and deployed by Google engineers, so it's still Google engineers browsing through people's private photos. Just because I use a tool doesn't mean I'm no longer responsible for my actions. It's exactly this complete dissociation of responsibility for the consequences why we will be facing an increasing anti-techie backlash. It's time we stopped hiding behind semi-autistic semantics. Google engineers went through people's private photos. Period. What tools they used to do it is utterly irrelevant.
- icebraining 12y ago
  A non-technical person will not consider that any different. That's not my experience at all. I stopped using Gmail in part due to privacy concerns, and when I explain it to non-techies, they're OK with it specifically because it's software and not people looking at the photos. People, in my experience, aren't worried about potential false positives or abuses of power. They just don't like the idea of other persons looking at their private pictures.
- JoshTriplett 12y ago
  This is not 4chan; "autistic" is not an appropriate description for anything other than the actual psychological disorder.
  - tomjen3 12y ago
    This is the internet. If you are going to be offended over the use of words you are going to have a bad time and should log of right now.
    - corpselord420 12y ago
      this is a website with a community. if you're going to prioritize some asshole's right to be a bigot over being inclusive then you're welcome to fuck right off.
    - AndrewDucker 12y ago
      This is a site on the internet. Which doesn't mean that it can't have standards of conduct that make it possible to have civilised discussion.
      - Pacabel 12y ago
        When read without putting political correctness above all else, the comment from bowlofpetunias is quite civilized. I think it brings up some very good points about how non-technical people and technical people can interpret situations involving technology in very different ways, for example. And I think it's actually trying to hold the technical community to a much higher standard when it comes to justifying actions that society at large will very likely disagree with. It's far more harmful and disruptive to see the perfectly legitimate argument of bowlofpetunias being totally ignored and even obscured, merely because some people find a single word in the comment to potentially be politically incorrect or potentially offensive. Discussion is much better when the focus is on the argument or topic itself, rather than going off on political correctness tangents over a single word in somebody's comment.
    - 27182818284 12y ago
- DanBC 12y ago
  > It's time we stopped hiding behind semi-autistic semantics. Semi-autistic is pretty fucking offensive. > Google engineers went through people's private photos. Period. What tools they used to do it is utterly irrelevant. Ignoring for a moment the fact that the whole point of Google is to inspect your traffic in order to serve you ads (and that's been used by competitors in their own adverts before; see Microsoft) there's a big difference between taking[1] image hashes to then compare to hashes of data passing through your network and some person sat at a desk looking at the images. [1] Probably being given image hashes by law enforcement. A much more interesting question is how fast they can do this? Imagine a set of say 500,000 hashes of the worst images. And global Google traffic. That's a lot of hashing and comparing going on. And are they doing simplistic hash checking, or is it more complicated so that people can't just change a few pixels to evade the hashes?
  - morsch 12y ago
    Wait wait wait, so it's acceptable now that government/LEO give a list of forbidden hashes to service providers with the expectation that they filter their networks and report anybody who's a match? Because that sounds bat shit insane. Who's to say protest-plans.ppt won't be the next thing on the list? Or justin-bieber.mp3? Or cia-torture-prisons.txt? With a hash, the ISP doesn't even know what the file is about before they detect a match, and having them look closer at matches and decide whether the government needs to know is just as insane.
    - meowface 12y ago
      >Who's to say protest-plans.ppt won't be the next thing on the list? Or justin-bieber.mp3? Or cia-torture-prisons.txt? Because possession of those files doesn't break federal law, but possession of child pornography does? And even if those things were added, obviously Google does some sort of rough verification process of the nature of the content being sent if it matches a hash. Service providers have been sharing image hashes of CP for 6+ years now. It's definitely not just Google. Your slippery slope makes no sense. This is probably the absolute least invasive thing that Google does. You are aware they show ads based on the actual text content of your emails, right?
Excavator 12y ago
On that topic: http://articles.latimes.com/2010/sep/16/business/la-fi-google-firing-20100916 http://articles.latimes.com/2010/sep/16/business/la-fi-googl... http://gawker.com/5637234/gcreep-google-engineer-stalked-teens-spied-on-chats http://gawker.com/5637234/gcreep-google-engineer-stalked-tee...
GeneralMayhem 12y ago
That's also what any non-technical person who thinks about for more than a millisecond would realize too, given the volume of Gmail traffic. Unfortunately, a millisecond's thought isn't conducive to fearmongering.
- dllthomas 12y ago
  A millisecond's thought will tell you that Google engineers aren't looking at every email (or a substantive fraction). That doesn't mean they aren't looking at any email, which is all that is necessary for that interpretation.
  - tptacek 12y ago
    The latter interpretation, if I understand it correctly, is that Google engineers randomly poke at people's emails and in this case just happened to get lucky. Even stipulating thousands of Google engineers doing that, that doesn't sound statistically plausible; the base rate of child pornography among Google Mail users is extremely low.
    - eloff 12y ago
      That was my thoughts as well. But I don't think most people think that way, especially given the misleading wording of the article. I hate to contribute to the "rest of the world is stupid" theory, or pretend that I'm immune from such things, but in this case I really do think the vast majority of people won't evaluate things in a probabilistic way like that. Most people just aren't trained to think in terms of odds instead of emotions (and it constantly annoys me.) At least that is my experience, and it may not be representative. Anecdote, I go swimming in a river known to have Caiman (small alligators) downstream and everyone thinks I'm a huge risk taker. Nobody comments on the 3 hour drive to get there and back in heavy rain and high speeds. It's obvious, mathematically speaking, which is the riskier activity.
    - dllthomas 12y ago
      I agree that it still seems unlikely, but it requires deeper thought to deduce that than "Google engineers can't possibly be looking at every email".

psykovsky 12y ago

PhotoDNA[0], for the ones interested in knowing a little bit more [0] http://en.wikipedia.org/wiki/PhotoDNA http://en.wikipedia.org/wiki/PhotoDNA

onewaystreet 12y ago
Anyone know of a similar technology (generating a hash from an image that is resistant to alterations) that is open source?
- krapp 12y ago
  If you don't mind php, I wrote this ages ago specifically to catch image spam on forums... the project is more or less dead now because i'm not technically capable of improving it, but it does work (albeit with false positives in some instances): https://github.com/kennethrapp/phasher https://github.com/kennethrapp/phasher You may find the actual algorithm used more interesting (http://phash.org/ http://phash.org/)
- troels 12y ago
  I'm fairly sure [OpenCV](http://opencv.org http://opencv.org) can do that for you.

dm2 12y ago

Here is a very simple overview of how it works: http://www.microsoft.com/global/en-us/news/publishingimages/ImageGallery/Images/Infographics/PhotoDNA/flowchart_photodna_Print.jpg http://www.microsoft.com/global/en-us/news/publishingimages/...

bcoates 12y ago

That's a nice, subtle way of fucking with someone: spoofing messages that trip these pedo-sensors won't result in an obvious mistaken raid, because the police will want something more solid to go on (like the case in the OP) -- the victim will just show up as 'pedophile, we just can't prove it yet' in a bunch of police databases/background checks/opposition research, and won't know why the random searches randomly choose them every time.

tptacek 12y ago
There's no such thing as a "pedo-sensor". This is a hash-based search against a corpus of known images, where the hash function was designed explicitly (a) to survive image transformations and (b) to prevent unauthorized parties from predicting the hash of a given image. Either of those two goals could fail and it would still be implausible that an attacker could trigger a false positive without using an image derived from actual known child pornography.
- true_religion 12y ago
  I think he means spoofing the message was sent by a particular person, but not actually spoofing the content. The content has to be real to trip the hash-search.
  - deleted 12y ago
    [deleted]
- dredmorbius 12y ago
  I'm actually aware of systems which were built to identify porn through neural-network training. I don't know if these were sensitive to distinguish child porn from other, but as a method of finding images likely to contain more skin than desired, they worked fairly reliably. That said, the Google/Microsoft tool does seem to work based on a known image corpus.
joeframbach 12y ago
Perhaps this may change if a certain list of 535 people in DC are added to this list.

doe88 12y ago

And what happens if someone (who wish you bad) anonymously send you such images as if he was your longtime buddy? Do they also report you?

deleted 12y ago
[deleted]

lifeisstillgood 12y ago

From the write up >> Since 2003, NCMEC has reviewed and analyzed almost 30 million images and videos of child pornography I deeply deeply hope that is not 30M distinct photos ... thats an industrial level output not a few ugly people trafficking in horror. I was struck by a recent podcast from BBCs More Or Less which tried to estimate number of paedophiles in population - and came up with 1%-5%. The problem was the definition - sexual thoughts, sexual actions with pre-pubescent of pubescent children upto the age of consent. That lumped people who thought Kate Moss' early nude photos were sexy together with people who rape 4 year olds which I don't think helps the debate. Anyway - I think the prevalence is greater than is generally recognised - but even so - 30 M images seems enormous.

dredmorbius 12y ago

The question of what Google might search for, using what hashes, on whom, why and when, and how, comes to mind. Google didn't "look at pictures". It had a database of known child porn images for which it computed hashes or signatures, which then could be checked, presumably of attachments in emails. This lets Google avoid the matter of "we looked at your specific images". It also means that your random personal photos aren't going to be examined (though their hashes are computed, and possibly retained). The hashing approach could apply to other aspects of email as well, though. Google could, perhaps, be encouraged to maintain a set of hashes corresponding to musical performances (it uses a similar technology AFAIU for YouTube copyright determination). Or of books. It could even take tuples of word sequences within email, compute hashes, and compare these with known word strings, forwarding relevant information to interested parties should certain sequences occur. All without "reading" your email. Where do you draw the line? Is it "think of the children"? Do we extend this to "think of the terrorists"? How about seeking out white-collar crime -- a set of account numbers, or key phrases, or transaction sequences? Or do we look for drugs crimes? Or gang activity? Or political insurrection? Can different national jurisdictions come up with their own sets of keywords? Say, those associated with specific religious or ethnic groups and activities? Uighurs, or Tibetan independence movement? Boko Haram? Aum Shinrikyo? Provisional IRA? Shining Path? ELF and PETA? Bangsamoro Islamic Freedom Fighters? Because those are all possible with slight variations on the methods described here. Or how should (or must) Google respond if users employ encryption to defeat such attempted detection?