8 ms·

Google tips off cops after spotting child abuse images in email

112 points by skreuzer 12y ago

reuwsaat 12y ago
Google's actions were legal. The problem is, we, as a country, have still not come to an agreement on what 'privacy' should entail in this brave new world. Given that Google was not breaking the law in reporting this person, I would argue it would have been immoral for them to have sat on this information.
- psykovsky 12y ago
  Google won't be breaking the law either when they ask them to report some other stuff that might make a target out of you. I say it's moral for them to report on you when the time comes. You've got nothing to hide, amirite?
  - pooper 12y ago
    How can Google and Microsoft comply with these requests and still claim that the emails they data held in Ireland is exempt? Moving on, what if blasphemy is illegal in another country and they (Google, Microsoft) spots someone who sent a a private message to someone else where they are making jokes about the prophet of Islam? What about countries ass-bent like India where you can't say anything bad about politicians? Will Google volunteer data about people who make bad jokes about politicians in private? I know it sounds ridiculous but you're right. Where does the slippery slope end? Problem is that on the other end... if they don't do this then they will have a PR nightmare. If people can spin their services as a safe haven for pedophiles, they will have a hard time.
    - reuwsaat 12y ago
      Did not know about those laws in India. That is mind-blowing! Not good. I personally agree with this line of logic. I'm working on a personal solution for moving my data off the cloud. But, that isn't a substitute for having this discussion as a country. Until we draw lines that slippery slope exists and these things are legal. EDIT: Regarding the safe-haven issue, I think that is a major issue with services like SpiderOak. They play up the ability to hide data. I personally believe we should be aiming for a solution that is closer to the locks on the doors of our homes. I feel secure at night, but I also know the police can knock it down by force if given a warrant. SpiderOak and other such services try to be more of a Fort Knox than a dead bolt. This is the cloud after all. Not a thumb drive in my safe in my home office.
      - pooper 12y ago
        About the safe haven issue, I'd imagine that if there is probable cause, they can just serve me a warrant rather than serving the guard who is on duty at Fort Knox. From what I know, it is wrong to bypass me to get to my data. If my data happens to reside outside the country, well tough luck. Transfer the case over to the other country and stop being a bully.
    - kumarm 12y ago
      Not sure why you are being down voted. Laws vary by country (Even if they are democratic for 50 year and more) and laws in some developing country appear stupid to any one in a developed country. US has Freedom of Speech. India has exact opposite. You can get into trouble because no matter what you do, you will offend someone.
    - true_religion 12y ago
      I don't think they're doing this to follow a particular law. They're doing it to follow their own internal ethics about child abuse. If they start reporting sedition and blasphemy, it'd mean that Google's ethics as a group has fundamentally changed, and we'd see far greater effects than their occasionally reporting someone.
- TehCorwiz 12y ago
  I do have one question. What if it wasn't child porn? Would it be right for them to go looking through your email for evidence of tax fraud? How about smoking weed? Where do you draw the line? The question isn't "do the ends justify the means?" the question should be "What if they're mistaken?" EDIT: While I stand by my questions I acknowledge the use of PhotoDNA as opposed to simply rifling through someone's inbox.
  - krapp 12y ago
    Is Google legally liable if it's discovered that they're hosting emails with child porn attached? If so, then does that suggest that Google owns the email hosted on their servers, and has the right to examine them if they want?
    - icebraining 12y ago
      I think Google's only liable if they know about it and didn't report. According to paragraph (f) of the 18 U.S. Code § 2258A, Protection of Privacy.— Nothing in this section shall be construed to require an electronic communication service provider or a remote computing service provider to— (1) monitor any user, subscriber, or customer of that provider; (2) monitor the content of any communication of any person described in paragraph (1); or (3) affirmatively seek facts or circumstances described in sections (a) and (b). http://www.law.cornell.edu/uscode/text/18/2258A?quicktabs_8=2#quicktabs-8 http://www.law.cornell.edu/uscode/text/18/2258A?quicktabs_8=...
  - lclarkmichalek 12y ago
    Your question doesn't seem to be "What if they're mistaken?"; it seems you are more asking "will they stop at child porn?". Anyway, I'd agree with jfoutz, in saying that society has dealt with these problems before, and has managed to create a fairly good line as to where something must be reported, and where a person's privacy becomes more important. That's not to say they will manage to create that line again, but it's also not to say they won't. Regardless, I don't think it's an unavoidable slippery slope. As for "What if they're mistaken?", well I highly doubt anyone will be convicted off the basis of an automated image recognition tool alone. If they are mistaken from time to time, then the person will probably have a warrant (ha) issued for their email, and any investigation would follow normally. If they're mistaken very often (i.e. the PhotoDNA implementation turns out to be shit), then the police (or whoever is receiving these reports) will probably stop caring, and nothing will have changed.
- jfoutz 12y ago
  Doctors and Lawyers are allowed and even obligated to break confidentiality in some special cases. We'll probably come up with a collection of special protections and special exceptions for remote storage of our data. Although, it's pretty scary that some random person could send an email that would then send me to jail. I would hope the standard for possession is higher than "WTF is this? [delete]"
  - icebraining 12y ago
    I would hope the standard for possession is higher than "WTF is this? [delete]" Well, according to the linked article, the guy was sending the picture, not receiving it.
    - saalweachter 12y ago
      I would still hope that "their email account sent an image" is only sufficient to start an investigation and not end it. Email accounts can be broken into, many people have access to the same computer, etc etc.
- chrismcb 12y ago
  Just because something is legal doesn't mean it is right nor that someone should do it. Yes, if Google knew about it they should do something. But Google shouldn't have known.
- deleted 12y ago
  [deleted]
patio11 12y ago
Google, Microsoft, and Facebook have been reported as having a list of image hashes which they very strongly do not want you sending through their systems. If you attempt to do so, that won't work out well for you. One writeup of many: http://www.microsoft.com/en-us/news/features/2009/dec09/12-15photodna.aspx http://www.microsoft.com/en-us/news/features/2009/dec09/12-1...
- eloff 12y ago
  That makes more sense, the article makes it sound like Google engineers were browsing through people's private photos, at least that's what every non-technical person who reads it will take away. It seems obvious to me that it must have been some kind of automated process that flagged the images.
  - bowlofpetunias 12y ago
    A non-technical person will not consider that any different. After all, said automated process was created and deployed by Google engineers, so it's still Google engineers browsing through people's private photos. Just because I use a tool doesn't mean I'm no longer responsible for my actions. It's exactly this complete dissociation of responsibility for the consequences why we will be facing an increasing anti-techie backlash. It's time we stopped hiding behind semi-autistic semantics. Google engineers went through people's private photos. Period. What tools they used to do it is utterly irrelevant.
    - icebraining 12y ago
      A non-technical person will not consider that any different. That's not my experience at all. I stopped using Gmail in part due to privacy concerns, and when I explain it to non-techies, they're OK with it specifically because it's software and not people looking at the photos. People, in my experience, aren't worried about potential false positives or abuses of power. They just don't like the idea of other persons looking at their private pictures.
    - JoshTriplett 12y ago
glynjackson 12y ago
When you use Google (Gmail) you signup to their terms of service. It states that incoming and outgoing emails are analysed by automated software. The article makes it seems like staff at Google are reading peoples emails or spying on its customers. They don't!
- lcedp 12y ago
  Usually they don't. [1] [1] http://gawker.com/5637234/gcreep-google-engineer-stalked-teens-spied-on-chats http://gawker.com/5637234/gcreep-google-engineer-stalked-tee...
  - DanBC 12y ago
    That case was very bad. It gets mentioned a lot. Is that because it happens all the time, but that's the only case that was reported? Or is it because it's very rare, and that's the only case there is to report? (I doubt very much that Google has only had one employee who accesses user data inappropriately and I really hope they take severe action when they find it. I would really like to know the rates from similar large companies - FB, MS, etc.)
    - lcedp 12y ago
      I'm not attacking Google in particular. It's just that the statement "Staff at [X] aren't reading peoples emails or spying on its customers" is simply logically false. Because sometimes they do it and we know it. Also, spying by the means of automatic analysis is still spying. (Also, I believe the reason it's done automatically is simply because it's more efficient, not ethics issues)
doctorKrieger 12y ago
"gentlemen don't read each other's mail" EOT,
higherpurpose 12y ago
This is why we need end-to-end encrypted communications to become mainstream. Why the hell are our service providers becoming Internet police these days? We didn't join Gmail or Facebook to "protect us from the bad guys", nor to have them turn people in for whatever arbitrary "serious crime" they decide they should support. It could be "child abuse" today, and "terrorism" (with whatever vague definition accompanying it) tomorrow. It's bad enough that the US government and Courts now think they have access to anyone's information in the world, if they use American services. But US service providers acting like they work for the government makes it even worse. Just stay out of our private communications. How hard can that be, really?
- DannyBee 12y ago
  " Why the hell are our service providers becoming Internet police these days?" Because laws keep getting passed that ask them to?
- DanBC 12y ago
  "These days" - this is nothing new. I'm genuinely confused by the number of people who think that any form of online communication is secure. I understand the outrage about government snooping. But this is a company checking its users do not break the law on the company's service and it is not in any way new. And having seen what idiot politicians would want I'm glad that services are doing this instead.
- tptacek 12y ago
  Really? Detection of known instances of child pornography is the reason we need end-to-end encryption? As an advocate for end-to-end security, I find this disquieting, in the "I think you're doing more harm than good with that argument" sense.
s_q_b 12y ago
Wait, what? Gmail is reading the content of email messages, and giving them to law enforcement? There are a bevy of constitutional issues with this. Now, I will preface this by saying that child molesters are despicable, no one will argue that. But because they are so vile, they make great test cases for pushing the boundaries of search and seizure law. But the problem is that once the courts create a legal rationale to justify Google reading the child molester's email, that rationale can be applied to anyone. So there's a whole host of defenses that could be raised: 1. The Wiretap Act - prohibits interception of electronic communications by private entity. Even Google's automated scanners, if applied to Gmail, could fall afoul of this law. 2. The Stored Communications Act - The Fourth Amendment creates very limited protections for stored electronic communications like email under the "third party doctrine." The third party doctrine essentially states that you have no privacy rights under the Fourth for any information you voluntarily send to another company or individual. The rationale being that, since you knowingly transmitted it, there is no reasonable expectation of privacy. However, The Stored Communications Act, part of ECPA, creates several statutory quasi-Fourth Amendment rights for stored electronic communications, especially email. 3. Agent of the State This is the weakest argument, but essentially the defendant could argue that Google was acting at the behest of the government, and thus full constitutional protections would attach. The rationale here is that Google and law enforcement were working so hand-in-glove that Google was a de facto state agent. This would make it very messy, as more powerful Constitutional protections would attach. Now, the bottom line here is that Google is reading content and transmitting it to law enforcement. I'm sure the right to read your email is in their ToS, but that could be successfully challenged if the language is either too narrow or overly broad as to be vague and thus void (two mistakes highly-paid attorneys make quite often.) EDIT: Forgot the exclusionary rule doesn't apply to evidence obtained illegally by private parties. You'd need to show the public-private nexus to have a chance in the criminal prosecution, but if you can brush Google back off the plate a bit using the private causes of action in #1 and #2, you may be able to get the discovery you need to prove the state-agent argument. For example, I guarantee Google has government Powerpoints floating around about activities certain agencies would like reported.
deleted 12y ago
[deleted]

112 points by skreuzer 12y ago

Google's actions were legal. The problem is, we, as a country, have still not come to an agreement on what 'privacy' should entail in this brave new world. Given that Google was not breaking the law in reporting this person, I would argue it would have been immoral for them to have sat on this information.

psykovsky 12y ago
Google won't be breaking the law either when they ask them to report some other stuff that might make a target out of you. I say it's moral for them to report on you when the time comes. You've got nothing to hide, amirite?
- pooper 12y ago
  How can Google and Microsoft comply with these requests and still claim that the emails they data held in Ireland is exempt? Moving on, what if blasphemy is illegal in another country and they (Google, Microsoft) spots someone who sent a a private message to someone else where they are making jokes about the prophet of Islam? What about countries ass-bent like India where you can't say anything bad about politicians? Will Google volunteer data about people who make bad jokes about politicians in private? I know it sounds ridiculous but you're right. Where does the slippery slope end? Problem is that on the other end... if they don't do this then they will have a PR nightmare. If people can spin their services as a safe haven for pedophiles, they will have a hard time.
  - reuwsaat 12y ago
    Did not know about those laws in India. That is mind-blowing! Not good. I personally agree with this line of logic. I'm working on a personal solution for moving my data off the cloud. But, that isn't a substitute for having this discussion as a country. Until we draw lines that slippery slope exists and these things are legal. EDIT: Regarding the safe-haven issue, I think that is a major issue with services like SpiderOak. They play up the ability to hide data. I personally believe we should be aiming for a solution that is closer to the locks on the doors of our homes. I feel secure at night, but I also know the police can knock it down by force if given a warrant. SpiderOak and other such services try to be more of a Fort Knox than a dead bolt. This is the cloud after all. Not a thumb drive in my safe in my home office.
    - pooper 12y ago
      About the safe haven issue, I'd imagine that if there is probable cause, they can just serve me a warrant rather than serving the guard who is on duty at Fort Knox. From what I know, it is wrong to bypass me to get to my data. If my data happens to reside outside the country, well tough luck. Transfer the case over to the other country and stop being a bully.
  - kumarm 12y ago
    Not sure why you are being down voted. Laws vary by country (Even if they are democratic for 50 year and more) and laws in some developing country appear stupid to any one in a developed country. US has Freedom of Speech. India has exact opposite. You can get into trouble because no matter what you do, you will offend someone.
  - true_religion 12y ago
    I don't think they're doing this to follow a particular law. They're doing it to follow their own internal ethics about child abuse. If they start reporting sedition and blasphemy, it'd mean that Google's ethics as a group has fundamentally changed, and we'd see far greater effects than their occasionally reporting someone.
TehCorwiz 12y ago
I do have one question. What if it wasn't child porn? Would it be right for them to go looking through your email for evidence of tax fraud? How about smoking weed? Where do you draw the line? The question isn't "do the ends justify the means?" the question should be "What if they're mistaken?" EDIT: While I stand by my questions I acknowledge the use of PhotoDNA as opposed to simply rifling through someone's inbox.
- krapp 12y ago
  Is Google legally liable if it's discovered that they're hosting emails with child porn attached? If so, then does that suggest that Google owns the email hosted on their servers, and has the right to examine them if they want?
  - icebraining 12y ago
    I think Google's only liable if they know about it and didn't report. According to paragraph (f) of the 18 U.S. Code § 2258A, Protection of Privacy.— Nothing in this section shall be construed to require an electronic communication service provider or a remote computing service provider to— (1) monitor any user, subscriber, or customer of that provider; (2) monitor the content of any communication of any person described in paragraph (1); or (3) affirmatively seek facts or circumstances described in sections (a) and (b). http://www.law.cornell.edu/uscode/text/18/2258A?quicktabs_8=2#quicktabs-8 http://www.law.cornell.edu/uscode/text/18/2258A?quicktabs_8=...
- lclarkmichalek 12y ago
  Your question doesn't seem to be "What if they're mistaken?"; it seems you are more asking "will they stop at child porn?". Anyway, I'd agree with jfoutz, in saying that society has dealt with these problems before, and has managed to create a fairly good line as to where something must be reported, and where a person's privacy becomes more important. That's not to say they will manage to create that line again, but it's also not to say they won't. Regardless, I don't think it's an unavoidable slippery slope. As for "What if they're mistaken?", well I highly doubt anyone will be convicted off the basis of an automated image recognition tool alone. If they are mistaken from time to time, then the person will probably have a warrant (ha) issued for their email, and any investigation would follow normally. If they're mistaken very often (i.e. the PhotoDNA implementation turns out to be shit), then the police (or whoever is receiving these reports) will probably stop caring, and nothing will have changed.
jfoutz 12y ago
Doctors and Lawyers are allowed and even obligated to break confidentiality in some special cases. We'll probably come up with a collection of special protections and special exceptions for remote storage of our data. Although, it's pretty scary that some random person could send an email that would then send me to jail. I would hope the standard for possession is higher than "WTF is this? [delete]"
- icebraining 12y ago
  I would hope the standard for possession is higher than "WTF is this? [delete]" Well, according to the linked article, the guy was sending the picture, not receiving it.
  - saalweachter 12y ago
    I would still hope that "their email account sent an image" is only sufficient to start an investigation and not end it. Email accounts can be broken into, many people have access to the same computer, etc etc.
chrismcb 12y ago
Just because something is legal doesn't mean it is right nor that someone should do it. Yes, if Google knew about it they should do something. But Google shouldn't have known.
deleted 12y ago
[deleted]

patio11 12y ago

Google, Microsoft, and Facebook have been reported as having a list of image hashes which they very strongly do not want you sending through their systems. If you attempt to do so, that won't work out well for you. One writeup of many: http://www.microsoft.com/en-us/news/features/2009/dec09/12-15photodna.aspx http://www.microsoft.com/en-us/news/features/2009/dec09/12-1...

eloff 12y ago
That makes more sense, the article makes it sound like Google engineers were browsing through people's private photos, at least that's what every non-technical person who reads it will take away. It seems obvious to me that it must have been some kind of automated process that flagged the images.
- bowlofpetunias 12y ago
  A non-technical person will not consider that any different. After all, said automated process was created and deployed by Google engineers, so it's still Google engineers browsing through people's private photos. Just because I use a tool doesn't mean I'm no longer responsible for my actions. It's exactly this complete dissociation of responsibility for the consequences why we will be facing an increasing anti-techie backlash. It's time we stopped hiding behind semi-autistic semantics. Google engineers went through people's private photos. Period. What tools they used to do it is utterly irrelevant.
  - icebraining 12y ago
    A non-technical person will not consider that any different. That's not my experience at all. I stopped using Gmail in part due to privacy concerns, and when I explain it to non-techies, they're OK with it specifically because it's software and not people looking at the photos. People, in my experience, aren't worried about potential false positives or abuses of power. They just don't like the idea of other persons looking at their private pictures.
  - JoshTriplett 12y ago

glynjackson 12y ago

When you use Google (Gmail) you signup to their terms of service. It states that incoming and outgoing emails are analysed by automated software. The article makes it seems like staff at Google are reading peoples emails or spying on its customers. They don't!

lcedp 12y ago
Usually they don't. [1] [1] http://gawker.com/5637234/gcreep-google-engineer-stalked-teens-spied-on-chats http://gawker.com/5637234/gcreep-google-engineer-stalked-tee...
- DanBC 12y ago
  That case was very bad. It gets mentioned a lot. Is that because it happens all the time, but that's the only case that was reported? Or is it because it's very rare, and that's the only case there is to report? (I doubt very much that Google has only had one employee who accesses user data inappropriately and I really hope they take severe action when they find it. I would really like to know the rates from similar large companies - FB, MS, etc.)
  - lcedp 12y ago
    I'm not attacking Google in particular. It's just that the statement "Staff at [X] aren't reading peoples emails or spying on its customers" is simply logically false. Because sometimes they do it and we know it. Also, spying by the means of automatic analysis is still spying. (Also, I believe the reason it's done automatically is simply because it's more efficient, not ethics issues)

doctorKrieger 12y ago

"gentlemen don't read each other's mail" EOT,

higherpurpose 12y ago

This is why we need end-to-end encrypted communications to become mainstream. Why the hell are our service providers becoming Internet police these days? We didn't join Gmail or Facebook to "protect us from the bad guys", nor to have them turn people in for whatever arbitrary "serious crime" they decide they should support. It could be "child abuse" today, and "terrorism" (with whatever vague definition accompanying it) tomorrow. It's bad enough that the US government and Courts now think they have access to anyone's information in the world, if they use American services. But US service providers acting like they work for the government makes it even worse. Just stay out of our private communications. How hard can that be, really?

DannyBee 12y ago
" Why the hell are our service providers becoming Internet police these days?" Because laws keep getting passed that ask them to?
DanBC 12y ago
"These days" - this is nothing new. I'm genuinely confused by the number of people who think that any form of online communication is secure. I understand the outrage about government snooping. But this is a company checking its users do not break the law on the company's service and it is not in any way new. And having seen what idiot politicians would want I'm glad that services are doing this instead.
tptacek 12y ago
Really? Detection of known instances of child pornography is the reason we need end-to-end encryption? As an advocate for end-to-end security, I find this disquieting, in the "I think you're doing more harm than good with that argument" sense.

s_q_b 12y ago

Wait, what? Gmail is reading the content of email messages, and giving them to law enforcement? There are a bevy of constitutional issues with this. Now, I will preface this by saying that child molesters are despicable, no one will argue that. But because they are so vile, they make great test cases for pushing the boundaries of search and seizure law. But the problem is that once the courts create a legal rationale to justify Google reading the child molester's email, that rationale can be applied to anyone. So there's a whole host of defenses that could be raised: 1. The Wiretap Act - prohibits interception of electronic communications by private entity. Even Google's automated scanners, if applied to Gmail, could fall afoul of this law. 2. The Stored Communications Act - The Fourth Amendment creates very limited protections for stored electronic communications like email under the "third party doctrine." The third party doctrine essentially states that you have no privacy rights under the Fourth for any information you voluntarily send to another company or individual. The rationale being that, since you knowingly transmitted it, there is no reasonable expectation of privacy. However, The Stored Communications Act, part of ECPA, creates several statutory quasi-Fourth Amendment rights for stored electronic communications, especially email. 3. Agent of the State This is the weakest argument, but essentially the defendant could argue that Google was acting at the behest of the government, and thus full constitutional protections would attach. The rationale here is that Google and law enforcement were working so hand-in-glove that Google was a de facto state agent. This would make it very messy, as more powerful Constitutional protections would attach. Now, the bottom line here is that Google is reading content and transmitting it to law enforcement. I'm sure the right to read your email is in their ToS, but that could be successfully challenged if the language is either too narrow or overly broad as to be vague and thus void (two mistakes highly-paid attorneys make quite often.) EDIT: Forgot the exclusionary rule doesn't apply to evidence obtained illegally by private parties. You'd need to show the public-private nexus to have a chance in the criminal prosecution, but if you can brush Google back off the plate a bit using the private causes of action in #1 and #2, you may be able to get the discovery you need to prove the state-agent argument. For example, I guarantee Google has government Powerpoints floating around about activities certain agencies would like reported.

deleted 12y ago

[deleted]