8 ms·

You're not seeing the other side of the coin - the huge amount of spam and abuse that such systems correctly identify and remove. If every abuser requests those

You're not seeing the other side of the coin - the huge amount of spam and abuse that such systems correctly identify and remove. If every abuser requests those explanations (which they will) there will be far more spam going around the Internet. Just think about the army of "Facebook content moderators" who were a popular topic on HN recently due to the concerns over their mental health. (I am offering no solutions here, for I know none)

fredgrott 6y ago
no if the AI can be used to automate the banning it can be used to provide the electronic news email of the rule violated.
- gambiting 6y ago
  Well their main argument against it is that if you don't tell scammers which rule exactly they are breaking they can't improve until the app is approved. But of course that hits normal customers too. It's the equivalent of arresting random people on the street and not telling them why - surely, innocent people will just get their lawyer to free them.
  - bildung 6y ago
    aka due process
_qulr 6y ago
> If every abuser requests those explanations (which they will) It's not a request, it's a requirement. If your account is suspended, you deserve an explanation. You should get one without having to request it. I'm not saying that companies shouldn't be able to suspend accounts temporarily. I'm simply saying that there needs to be a way to get your account unsuspended if you're innocent. The way it "works" now is that innocent consumers are without any recourse whatsoever.
- daemin 6y ago
  I heard on a podcast recently that a trading system needs to keep logs of why a particular trade was executed for several years just in case the authority wants it. So it isn't too much effort to build a similar report or log of behaviour to explain why someone was banned. Obviously this will also help the spammers who will use this information to get around the filters.
  - daemin 6y ago
    Podcast link: https://www.twoscomplement.org/ https://www.twoscomplement.org/, I think it was the latest episode.
  - gibspaulding 6y ago
    Complete speculation because I don't actually know how this works, but I wonder if the explanation would be something like this: "You've been banned because our black box ML algorithm says your usage patterns share similar traits to those of known spammers."
    - p410n3 6y ago
      Thats kinda what the PayPal support told me when I asked why half of my in-store payments via Google Pay get rejected. Most were payments of about 2€ in the same store next to work. Whatever I dont use it anymore
    - BlueTemplar 6y ago
      Some government decisions are indirectly forbidden from using black box "algorithms" because they are obligated by law to explain (on demand) the steps that the algorithm took to reach its decision. Maybe something like this should also apply to some private companies ?
- AshamedCaptain 6y ago
  For the record, they don't give away these explanations because such explanation would hint the spammer to what they should _not_ do next time, to avoid getting caught. Same as with anticheat software.
_pmf_ 6y ago
> the huge amount of spam and abuse that such systems correctly identify and remove. Maybe allowing single service providers to capture several billions of users is the problem here.
- jeegsy 6y ago
  Ding Ding Ding!!!
paulsutter 6y ago
Perhaps the process should cost $100 or $500, so that actual spammers can’t use it Maybe they really just need to offer a paid account option with real support, since that has much better incentives
- wcoenen 6y ago
  There is a paid option: for $6/month you can use gmail with your own domain name. It's targeted at businesses but you can use it as an individual. https://workspace.google.com/pricing.html https://workspace.google.com/pricing.html It includes support, but I'm not sure if that helps in cases where google thinks you have abused the service. I just use it because I like having my own domain, and so that I don't lose access to my email if google locks me out. The idea is that I can update my domain's MTX records and use another email service.
  - sam_goody 6y ago
    Support does not include if your account gets suspended or if you lose access. We had a paid Google App account. One of our workers would only login from their computer. It died, and she tried to login from the new computer. It gave a unrecognized machine error, and we had to hire someone to resuscitate the old computer for her. I know of a company that had the entire companies' accounts suspended without warning because one user did something that violated their terms, but they could not figure out what. The company lost three months of revenue from it and I am not sure if it caused bankruptcy. No help at all from G.
    - KMag 6y ago
      About a year ago, I started migrating to a vanity domain, currently hosted at Google, for this reason. If I get locked out of Google, I lose my history, but at least I can move to another provider and avoid being locked out of my life for the indefinite future.
      - thw0rted 6y ago
        Friendly reminder to anybody reading this with a Google account: it's not a perfect solution, but head over to Google Takeout and grab a dump of your account data while you're thinking of it. I did one last year, and at the same time reconfigured my phone camera roll to back up somewhere outside the Google ecosystem, so now all I'm missing is an up to date email mirror.
khawkins 6y ago
I think this is a convenient narrative for an abusive pattern of behavior by Google. The company is infamous for having non-existent customer service. It's not a matter of their AI having too many false positives, it's that when there is a false positive you have literally no recourse even if you're a well known business partner. Are we really going to believe that Google, one of the highest grossing companies in the world, doesn't have the money to provide even basic level customer service? If it were really a matter of not being able to afford it, certainly they could offer it for a fee. No, they're stubbornly refusing to address the issues, relying on this lie, and using their market dominance to avoid having to answer for it.
- criley2 6y ago
  Technically they do offer customer service if you pay them with their Google One product. I have phone numbers and human access very quickly, because I pay for it. Although obviously if they banned me, I wouldn't have access to my direct support line anymore.
  - oauea 6y ago
    > Although obviously if they banned me, I wouldn't have access to my direct support line anymore. Which they will do literally on a whim. Who are you going to call then?
    - criley2 6y ago
      I reject that they ban "literally on a whim". I get that people do get banned, but clearly they aren't in the business of banning people based even on the number of people in this very thread with Google accounts. You don't have to exaggerate to make a point. (And now I'm being downvoted for pointing out the Google One support product. What a lovely community we've got here!)
      - oauea 6y ago
        > on a whim: because of a sudden decision How was them banning Terraria's accounts not a sudden decision? How about any of the other stories posted in this thread? They literally ban accounts on a whim, usually with no warnings issued.
syshum 6y ago
So what is the proper Blackstone's ratio for you in these situation? Is 1000 innocents ok to punish as long as 1 spam message is stopped?
ballenf 6y ago
This is an age old problem in the criminal justice system. A solved problem. After a lot of trials with various approaches, we settled on letting some criminals go free over convicting someone on weak evidence. Second we decided that trials should be open and evidence viewable by default. Finally you generally have the option to give some security to stay out of jail during trial. Closing a google account is a punishment worse than many criminal convictions. And will only get more important as we progress to an all digital existence.
mschuster91 6y ago
> Just think about the army of "Facebook content moderators" who were a popular topic on HN recently due to the concerns over their mental health. Hire them directly instead of via labor farms, pay them an actual living wage, give them full health benefits, and hire enough of them to prevent overload.
jlmb 6y ago
But surely it’s possible to use methods other than what currently seems to be the first and only solution: “your account has been banned, bye”. For example, if an automated system thinks an account is sending spam, enforcing a (very low) outgoing email rate limit would be a much more reasonable first step.
Chris2048 6y ago
So just start charging for service, and keep a non-refundable deposit for spam/abuse. Let every abuser requests those explanations, if the decision doesn't change, the money is still kept, which funds that service.

fredgrott 6y ago

no if the AI can be used to automate the banning it can be used to provide the electronic news email of the rule violated.

gambiting 6y ago
Well their main argument against it is that if you don't tell scammers which rule exactly they are breaking they can't improve until the app is approved. But of course that hits normal customers too. It's the equivalent of arresting random people on the street and not telling them why - surely, innocent people will just get their lawyer to free them.
- bildung 6y ago
  aka due process

_qulr 6y ago

> If every abuser requests those explanations (which they will) It's not a request, it's a requirement. If your account is suspended, you deserve an explanation. You should get one without having to request it. I'm not saying that companies shouldn't be able to suspend accounts temporarily. I'm simply saying that there needs to be a way to get your account unsuspended if you're innocent. The way it "works" now is that innocent consumers are without any recourse whatsoever.

daemin 6y ago
I heard on a podcast recently that a trading system needs to keep logs of why a particular trade was executed for several years just in case the authority wants it. So it isn't too much effort to build a similar report or log of behaviour to explain why someone was banned. Obviously this will also help the spammers who will use this information to get around the filters.
- daemin 6y ago
  Podcast link: https://www.twoscomplement.org/ https://www.twoscomplement.org/, I think it was the latest episode.
- gibspaulding 6y ago
  Complete speculation because I don't actually know how this works, but I wonder if the explanation would be something like this: "You've been banned because our black box ML algorithm says your usage patterns share similar traits to those of known spammers."
  - p410n3 6y ago
    Thats kinda what the PayPal support told me when I asked why half of my in-store payments via Google Pay get rejected. Most were payments of about 2€ in the same store next to work. Whatever I dont use it anymore
  - BlueTemplar 6y ago
    Some government decisions are indirectly forbidden from using black box "algorithms" because they are obligated by law to explain (on demand) the steps that the algorithm took to reach its decision. Maybe something like this should also apply to some private companies ?
AshamedCaptain 6y ago
For the record, they don't give away these explanations because such explanation would hint the spammer to what they should _not_ do next time, to avoid getting caught. Same as with anticheat software.

_pmf_ 6y ago

> the huge amount of spam and abuse that such systems correctly identify and remove. Maybe allowing single service providers to capture several billions of users is the problem here.

jeegsy 6y ago
Ding Ding Ding!!!

paulsutter 6y ago

Perhaps the process should cost $100 or $500, so that actual spammers can’t use it Maybe they really just need to offer a paid account option with real support, since that has much better incentives

wcoenen 6y ago
There is a paid option: for $6/month you can use gmail with your own domain name. It's targeted at businesses but you can use it as an individual. https://workspace.google.com/pricing.html https://workspace.google.com/pricing.html It includes support, but I'm not sure if that helps in cases where google thinks you have abused the service. I just use it because I like having my own domain, and so that I don't lose access to my email if google locks me out. The idea is that I can update my domain's MTX records and use another email service.
- sam_goody 6y ago
  Support does not include if your account gets suspended or if you lose access. We had a paid Google App account. One of our workers would only login from their computer. It died, and she tried to login from the new computer. It gave a unrecognized machine error, and we had to hire someone to resuscitate the old computer for her. I know of a company that had the entire companies' accounts suspended without warning because one user did something that violated their terms, but they could not figure out what. The company lost three months of revenue from it and I am not sure if it caused bankruptcy. No help at all from G.
  - KMag 6y ago
    About a year ago, I started migrating to a vanity domain, currently hosted at Google, for this reason. If I get locked out of Google, I lose my history, but at least I can move to another provider and avoid being locked out of my life for the indefinite future.
    - thw0rted 6y ago
      Friendly reminder to anybody reading this with a Google account: it's not a perfect solution, but head over to Google Takeout and grab a dump of your account data while you're thinking of it. I did one last year, and at the same time reconfigured my phone camera roll to back up somewhere outside the Google ecosystem, so now all I'm missing is an up to date email mirror.

khawkins 6y ago

I think this is a convenient narrative for an abusive pattern of behavior by Google. The company is infamous for having non-existent customer service. It's not a matter of their AI having too many false positives, it's that when there is a false positive you have literally no recourse even if you're a well known business partner. Are we really going to believe that Google, one of the highest grossing companies in the world, doesn't have the money to provide even basic level customer service? If it were really a matter of not being able to afford it, certainly they could offer it for a fee. No, they're stubbornly refusing to address the issues, relying on this lie, and using their market dominance to avoid having to answer for it.

criley2 6y ago
Technically they do offer customer service if you pay them with their Google One product. I have phone numbers and human access very quickly, because I pay for it. Although obviously if they banned me, I wouldn't have access to my direct support line anymore.
- oauea 6y ago
  > Although obviously if they banned me, I wouldn't have access to my direct support line anymore. Which they will do literally on a whim. Who are you going to call then?
  - criley2 6y ago
    I reject that they ban "literally on a whim". I get that people do get banned, but clearly they aren't in the business of banning people based even on the number of people in this very thread with Google accounts. You don't have to exaggerate to make a point. (And now I'm being downvoted for pointing out the Google One support product. What a lovely community we've got here!)
    - oauea 6y ago
      > on a whim: because of a sudden decision How was them banning Terraria's accounts not a sudden decision? How about any of the other stories posted in this thread? They literally ban accounts on a whim, usually with no warnings issued.

syshum 6y ago

So what is the proper Blackstone's ratio for you in these situation? Is 1000 innocents ok to punish as long as 1 spam message is stopped?

ballenf 6y ago

This is an age old problem in the criminal justice system. A solved problem. After a lot of trials with various approaches, we settled on letting some criminals go free over convicting someone on weak evidence. Second we decided that trials should be open and evidence viewable by default. Finally you generally have the option to give some security to stay out of jail during trial. Closing a google account is a punishment worse than many criminal convictions. And will only get more important as we progress to an all digital existence.

mschuster91 6y ago

> Just think about the army of "Facebook content moderators" who were a popular topic on HN recently due to the concerns over their mental health. Hire them directly instead of via labor farms, pay them an actual living wage, give them full health benefits, and hire enough of them to prevent overload.

jlmb 6y ago

But surely it’s possible to use methods other than what currently seems to be the first and only solution: “your account has been banned, bye”. For example, if an automated system thinks an account is sending spam, enforcing a (very low) outgoing email rate limit would be a much more reasonable first step.

Chris2048 6y ago

So just start charging for service, and keep a non-refundable deposit for spam/abuse. Let every abuser requests those explanations, if the decision doesn't change, the money is still kept, which funds that service.