5 ms·

They also conveniently forward to law enforcement parents who send non-sexual explicit images taken expressly for the purpose of examination by licensed medica

They also conveniently forward to law enforcement parents who send non-sexual explicit images taken expressly for the purpose of examination by licensed medical personnel. https://www.theguardian.com/technology/2022/aug/22/google-csam-account-blocked https://www.theguardian.com/technology/2022/aug/22/google-cs...

danpalmer 4y ago
I've read a bunch of articles about this and I just can't see how there isn't more to the story. Whether the original ban was automated or not, one of their specialists must have been involved by now, not least because of the media coverage and the fact they reported it to the police. While the police declined to investigate further (from what we know, last time I read about this), there must be another reason for the account ban to remain in place. I'm biased, I work at Google (I don't know anything about this that hasn't been reported), but I also just don't believe that a company would wilfully ban accounts that they don't have a good reason to ban, after human review. It would be a stupid move, especially considering the media attention. Hence my guess that there must be more to this story. Again I have no insider knowledge, this is purely personal speculation.
- dclusin 4y ago
  Multiple independent organizations inside a company each with their own rigid and inflexible operating procedures. The user to moderator ratio in the tens of thousands to 1. How is this hard to see?
  - kahrl 4y ago
    "I'm biased, I work at Google"
  - danpalmer 4y ago
    I don't know what you mean by the multiple organizations thing – this seems to be squarely in the realm of trust and safety. And while I understand the frustration in contacting a human sometimes, in this case it seems that a human will have reviewed the content and come to the conclusion based on their training, expert opinions, policies, etc (as per the article). What can human contact add here? Excuses or reasoning for the content? Given the media attention, someone will almost certainly have reached out if there was any way that a reason or excuse could possibly change the outcome. If they didn't that implies again that there's more to this story. I'm not going to pass any judgement on what the "more to the story" is here, I don't have all the facts, but it seems obvious to me that there are more facts, and that people are working from different amounts of information here.
    - dclusin 4y ago
      The two pieces of information I think that stand out are: 1. Law enforcement elected not to pursue charges 2. The banned parties were comfortable enough going to a respected newspaper with the facts of their case, which, the newspaper presumably also did due diligence and sided with the users. I don’t think that there’s really any “hidden” context here that would convince many (except maybe those who get their paychecks from big G) that google is not in the wrong. Perhaps the hidden context here is that they also investigated the couple after the journal report and found other “violations” which were ban worthy after, not before, their case was aired publicly. I don’t think many people would consider this theoretical scenario fair either.
      - lupire 4y ago
        10ish years ago there was a big kerfuffle about a semi famous author who got his account closed, and then it turned out that he was semi famous for...writing fictional child porn. Some people argue they fictional / textual child porn should be legal, but Google would never allow that in their servers. No idea about this case, but there are grey areas and the cowardly position is the legally/socially safest one, regardless of what one local police department decided. It doesn't make Google right, but it is what it is.
- wmf 4y ago
  Policy #0: Never admit a mistake.
- desmosxxx 4y ago
  There are possible legal reasons they wouldn't reverse an account ban of this sort. There are also several instances of Google employees or their spouses being banned and AFAIK none have led to any criminal charges - e.g. https://twitter.com/miguelytob/status/1315749803041619981 https://twitter.com/miguelytob/status/1315749803041619981 (not sure if this one was for csam or not - but IIRC some were). Police salivate over cases like this - until they decide to prosecute, it seems obvious that Google is in the wrong here.
  - lupire 4y ago
    No large business has ever said "I'll let my customers do whatever they want until the police forcibly stop them". It's too risky and unsavory. They set up a buffer around the law. Even legal adult porn is enough to get in trouble with Google.
- saurik 4y ago
  > It would be a stupid move... Please do remember that this company you work for is somewhat notorious on this website for making "stupid moves" seemingly every other day with respect to basic product management. The usual defense is that some internal incentive mechanism blindly optimized for these stupid moves and no one is at the wheel to change the company's high-level behavior... is there really any reason to believe things are different here? Is there a mechanism by which someone helping a user get around a CSAM filter would get them promoted at Google, or would they just be taking on potentially-arbitrary levels of internal risk going off the rails on a side project wherein they have to condone something that has an air of danger to it with the only real goal of helping a user who probably wasn't even paying Google anything in the first place? I admittedly don't work at Google, but the impression one gets on the outside looking in is that doing something that wouldn't cause some major impact on your performance review "isn't very Googley" :(.
dclusin 4y ago
And don’t restore accounts even when law enforcement decides not to pursue charges.
- 7952 4y ago
  That is expecting Google to prove a negative about the user. They probably don't want to have the obligation to verify that the user really is innocent. And police inaction is not evidence of that.
deleted 4y ago
[deleted]

danpalmer 4y ago

I've read a bunch of articles about this and I just can't see how there isn't more to the story. Whether the original ban was automated or not, one of their specialists must have been involved by now, not least because of the media coverage and the fact they reported it to the police. While the police declined to investigate further (from what we know, last time I read about this), there must be another reason for the account ban to remain in place. I'm biased, I work at Google (I don't know anything about this that hasn't been reported), but I also just don't believe that a company would wilfully ban accounts that they don't have a good reason to ban, after human review. It would be a stupid move, especially considering the media attention. Hence my guess that there must be more to this story. Again I have no insider knowledge, this is purely personal speculation.

dclusin 4y ago
Multiple independent organizations inside a company each with their own rigid and inflexible operating procedures. The user to moderator ratio in the tens of thousands to 1. How is this hard to see?
- kahrl 4y ago
  "I'm biased, I work at Google"
- danpalmer 4y ago
  I don't know what you mean by the multiple organizations thing – this seems to be squarely in the realm of trust and safety. And while I understand the frustration in contacting a human sometimes, in this case it seems that a human will have reviewed the content and come to the conclusion based on their training, expert opinions, policies, etc (as per the article). What can human contact add here? Excuses or reasoning for the content? Given the media attention, someone will almost certainly have reached out if there was any way that a reason or excuse could possibly change the outcome. If they didn't that implies again that there's more to this story. I'm not going to pass any judgement on what the "more to the story" is here, I don't have all the facts, but it seems obvious to me that there are more facts, and that people are working from different amounts of information here.
  - dclusin 4y ago
    The two pieces of information I think that stand out are: 1. Law enforcement elected not to pursue charges 2. The banned parties were comfortable enough going to a respected newspaper with the facts of their case, which, the newspaper presumably also did due diligence and sided with the users. I don’t think that there’s really any “hidden” context here that would convince many (except maybe those who get their paychecks from big G) that google is not in the wrong. Perhaps the hidden context here is that they also investigated the couple after the journal report and found other “violations” which were ban worthy after, not before, their case was aired publicly. I don’t think many people would consider this theoretical scenario fair either.
    - lupire 4y ago
      10ish years ago there was a big kerfuffle about a semi famous author who got his account closed, and then it turned out that he was semi famous for...writing fictional child porn. Some people argue they fictional / textual child porn should be legal, but Google would never allow that in their servers. No idea about this case, but there are grey areas and the cowardly position is the legally/socially safest one, regardless of what one local police department decided. It doesn't make Google right, but it is what it is.
wmf 4y ago
Policy #0: Never admit a mistake.
desmosxxx 4y ago
There are possible legal reasons they wouldn't reverse an account ban of this sort. There are also several instances of Google employees or their spouses being banned and AFAIK none have led to any criminal charges - e.g. https://twitter.com/miguelytob/status/1315749803041619981 https://twitter.com/miguelytob/status/1315749803041619981 (not sure if this one was for csam or not - but IIRC some were). Police salivate over cases like this - until they decide to prosecute, it seems obvious that Google is in the wrong here.
- lupire 4y ago
  No large business has ever said "I'll let my customers do whatever they want until the police forcibly stop them". It's too risky and unsavory. They set up a buffer around the law. Even legal adult porn is enough to get in trouble with Google.
saurik 4y ago
> It would be a stupid move... Please do remember that this company you work for is somewhat notorious on this website for making "stupid moves" seemingly every other day with respect to basic product management. The usual defense is that some internal incentive mechanism blindly optimized for these stupid moves and no one is at the wheel to change the company's high-level behavior... is there really any reason to believe things are different here? Is there a mechanism by which someone helping a user get around a CSAM filter would get them promoted at Google, or would they just be taking on potentially-arbitrary levels of internal risk going off the rails on a side project wherein they have to condone something that has an air of danger to it with the only real goal of helping a user who probably wasn't even paying Google anything in the first place? I admittedly don't work at Google, but the impression one gets on the outside looking in is that doing something that wouldn't cause some major impact on your performance review "isn't very Googley" :(.

dclusin 4y ago

And don’t restore accounts even when law enforcement decides not to pursue charges.

7952 4y ago
That is expecting Google to prove a negative about the user. They probably don't want to have the obligation to verify that the user really is innocent. And police inaction is not evidence of that.

deleted 4y ago

[deleted]