6 ms·

I don’t see how you get from Google’s statement “Was taken down for legal reasons and cannot be appealed“ to “no human was involved”.

kelnos 5y ago
I think the point is they, based on the file content, no human could have been involved in the decision. If there was a human involved, the files never would have been flagged.
- SAI_Peregrinus 5y ago
  You're assuming the human both has agency, and gives a damn. It's more likely the human just rubber-stamps all bans, to get their KPI of number of appeals processed per day up!
  - foxfluff 5y ago
    They are required to have agency and give a damn. Of course, it is hard to (dis)prove that they actually do.
  - saghm 5y ago
    If the human doesn't have agency, then it's not really a "human review", is it?
    - rat9988 5y ago
      It still is.
      - foxfluff 5y ago
        Spirit of the law is a concept I would encourage anyone to think about when arguing about these things. I believe most people, and courts in particular, would not agree that a human rubber-stamping automated decision is in line with the spirit of the law. Clinging onto a technicality isn't going to go well. I'd also like to point out that these laws don't just come out of nowhere in a vacuum, to be interpreted without any further context. In EU we have recitals and guidelines to give context and support the interpretation of regulations. If you're interested, do read Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679 (wp251rev.01). https://ec.europa.eu/newsroom/article29/items/612053/en https://ec.europa.eu/newsroom/article29/items/612053/en Here's what it says about human intervention: "Any review must be carried out by someone who has the appropriate authority and capability to change the decision. The reviewer should undertake a thorough assessment of all the relevant data, including any additional information provided by the data subject."
        SpicyLemonZest 5y ago
        But the entire premise here is a "letter of the law" thing. Online account bans are pretty clearly not within the spirit of the GDPR restrictions on automated decisionmaking; note how the guidelines you linked, despite providing quite a bit of detail about different kinds of automated decisionmaking and rules around them, don't mention account bans at all.
        foxfluff 5y ago
        There's only a handful of examples, and to me it is far from clear whether account bans would be in scope of the law. It's not meant to be an exhaustive list of all the things that are covered. However, I could make the case that losing an account which holds years of your private correspondence and is your point of contact for private exchange, services you rely on (including where bills, account recovery emails, policy changes, warnings & alerts, 2fa codes, and other very important messages are sent), potential employers or clients, and which doubles as a login for other services (see openid) and so on, can have a significant effect on your life and could potentially fall under "decisions that deny someone an employment opportunity or put them at a serious disadvantage" or (admittedly vague) "lead to the exclusion or discrimination of individuals." Some of the other examples in the guidelines seem mild by comparison (e.g. getting a reduced limit on credit card). My perspective is colored by both having lost access to an email account and also being denied a credit card application; the former was a much bigger problem.
      - bsedlm 5y ago
        it may still be human, but if it's just rubber stamping is it really a review?
      - philipov 5y ago
        And you've just demonstrated why programmatic enforcement of contracts doesn't work. Courts are actually able to see through that semantic nonsense, because humans are able to discern intent.
      - saghm 5y ago
        I think the chances that a judge would accept that argument is potentially a lot lower than you might expect.
      - throwaway22032 5y ago
        Here's a human review process for you: 1) Check whether the output of the machine learning model outputs Yes or No. 2) If yes, ban. 3) If no, no ban. This is not a human review process. The review process is algorithmic, the human is only involved to relay the result.
  - mdoms 5y ago
    That's an automated process then.
- Someone 5y ago
  The discussion isn’t about the initial flagging, it’s about the review. That might be as simple as checking for the existence of legal documents claiming copyright infringement, or as reading a web page stating “we already removed X other copies of this file”. Neither is a fail-safe way of doing such a review, but doing a thorough review might be expensive even for Google. Does anybody know how many such reviews they do each day? It might also be a bug on their tooling to assist human reviewers.
  - kelnos 5y ago
    Right, but I believe for those cases, the (automated) email also said that the ability to appeal was not available. So there is no human review, because there's no review at all. Regardless -- and I know this is a "how the world should be, not how it is" type thing -- I really think the initial decision should not be allowed to be made by an algorithm. At the very most, an algorithm should be allowed to flag something for human review, but no action is taken until the human has a chance to review it and decide if the flag is warranted or not.
deleted 5y ago
[deleted]
tyingq 5y ago
Because once a human did get involved, via noise from here and twitter, Google admitted that there was a problem. Also, just the absurdity that a human would review a file containing only "1" and decide the decision to flag it was correct. https://twitter.com/googledrive/status/1486038872928792576 https://twitter.com/googledrive/status/1486038872928792576
- godelski 5y ago
  What's interesting is there's another user there that followed up 2 days after the tweet noting that other numbers weren't fixed. But this is ignored. Problems shouldn't get fixed just because they got enough likes and reshares on Twitter.

kelnos 5y ago

I think the point is they, based on the file content, no human could have been involved in the decision. If there was a human involved, the files never would have been flagged.

SAI_Peregrinus 5y ago
You're assuming the human both has agency, and gives a damn. It's more likely the human just rubber-stamps all bans, to get their KPI of number of appeals processed per day up!
- foxfluff 5y ago
  They are required to have agency and give a damn. Of course, it is hard to (dis)prove that they actually do.
- saghm 5y ago
  If the human doesn't have agency, then it's not really a "human review", is it?
  - rat9988 5y ago
    It still is.
    - foxfluff 5y ago
      Spirit of the law is a concept I would encourage anyone to think about when arguing about these things. I believe most people, and courts in particular, would not agree that a human rubber-stamping automated decision is in line with the spirit of the law. Clinging onto a technicality isn't going to go well. I'd also like to point out that these laws don't just come out of nowhere in a vacuum, to be interpreted without any further context. In EU we have recitals and guidelines to give context and support the interpretation of regulations. If you're interested, do read Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679 (wp251rev.01). https://ec.europa.eu/newsroom/article29/items/612053/en https://ec.europa.eu/newsroom/article29/items/612053/en Here's what it says about human intervention: "Any review must be carried out by someone who has the appropriate authority and capability to change the decision. The reviewer should undertake a thorough assessment of all the relevant data, including any additional information provided by the data subject."
      - SpicyLemonZest 5y ago
        But the entire premise here is a "letter of the law" thing. Online account bans are pretty clearly not within the spirit of the GDPR restrictions on automated decisionmaking; note how the guidelines you linked, despite providing quite a bit of detail about different kinds of automated decisionmaking and rules around them, don't mention account bans at all.
        foxfluff 5y ago
        There's only a handful of examples, and to me it is far from clear whether account bans would be in scope of the law. It's not meant to be an exhaustive list of all the things that are covered. However, I could make the case that losing an account which holds years of your private correspondence and is your point of contact for private exchange, services you rely on (including where bills, account recovery emails, policy changes, warnings & alerts, 2fa codes, and other very important messages are sent), potential employers or clients, and which doubles as a login for other services (see openid) and so on, can have a significant effect on your life and could potentially fall under "decisions that deny someone an employment opportunity or put them at a serious disadvantage" or (admittedly vague) "lead to the exclusion or discrimination of individuals." Some of the other examples in the guidelines seem mild by comparison (e.g. getting a reduced limit on credit card). My perspective is colored by both having lost access to an email account and also being denied a credit card application; the former was a much bigger problem.
    - bsedlm 5y ago
      it may still be human, but if it's just rubber stamping is it really a review?
    - philipov 5y ago
      And you've just demonstrated why programmatic enforcement of contracts doesn't work. Courts are actually able to see through that semantic nonsense, because humans are able to discern intent.
    - saghm 5y ago
      I think the chances that a judge would accept that argument is potentially a lot lower than you might expect.
    - throwaway22032 5y ago
      Here's a human review process for you: 1) Check whether the output of the machine learning model outputs Yes or No. 2) If yes, ban. 3) If no, no ban. This is not a human review process. The review process is algorithmic, the human is only involved to relay the result.
- mdoms 5y ago
  That's an automated process then.
Someone 5y ago
The discussion isn’t about the initial flagging, it’s about the review. That might be as simple as checking for the existence of legal documents claiming copyright infringement, or as reading a web page stating “we already removed X other copies of this file”. Neither is a fail-safe way of doing such a review, but doing a thorough review might be expensive even for Google. Does anybody know how many such reviews they do each day? It might also be a bug on their tooling to assist human reviewers.
- kelnos 5y ago
  Right, but I believe for those cases, the (automated) email also said that the ability to appeal was not available. So there is no human review, because there's no review at all. Regardless -- and I know this is a "how the world should be, not how it is" type thing -- I really think the initial decision should not be allowed to be made by an algorithm. At the very most, an algorithm should be allowed to flag something for human review, but no action is taken until the human has a chance to review it and decide if the flag is warranted or not.

deleted 5y ago

[deleted]

tyingq 5y ago

Because once a human did get involved, via noise from here and twitter, Google admitted that there was a problem. Also, just the absurdity that a human would review a file containing only "1" and decide the decision to flag it was correct. https://twitter.com/googledrive/status/1486038872928792576 https://twitter.com/googledrive/status/1486038872928792576

godelski 5y ago
What's interesting is there's another user there that followed up 2 days after the tweet noting that other numbers weren't fixed. But this is ignored. Problems shouldn't get fixed just because they got enough likes and reshares on Twitter.