15 ms·

I think this is the future rise of the robots. It won't be Skynet, with clearly malicious bots trying to destroy humanity. It will be capricious and arbitrary

I think this is the future rise of the robots. It won't be Skynet, with clearly malicious bots trying to destroy humanity. It will be capricious and arbitrary bots, locking us out of our bank accounts and merchant accounts with little or no recompense. We've already seen it with Google, now here with Amazon. How soon until your phone gets turned off, and that's just the end?

dingaling 8y ago
In the past we spread our patronage and investments around to protect against the failure of businesses. Looks like in the future we'll do so to protect ourselves against AI.
- ConceptJunkie 8y ago
  Maybe there will be a flipside to this. Perhaps we can hope for the ability to have our own AIs that work on our behalf.
  - waisbrot 8y ago
    Maybe if the story author had written a bot to reply "I don't understand. Please tell me what I did wrong." to every bot-generated email from Amazon, he could have made a more entertaining story. He could include some graphs of the reply-counts, estimate the AZ bot's latency, and draw out some maps of the various routes taken by the messages.
kevin_b_er 8y ago
And if you're in the US, there are no restrictions on corporations restricting your rights. Only the government. The more that is privatized, the less rights you have.
- Finnucane 8y ago
  >And if you're in the US, there are no restrictions on corporations restricting your rights. That's not precisely true, though in practice it may depend on the willingness of regulators to enforce existing rules, or people being willing and able to sue in court. Our corporate overlords can often count on those factors working in their favor.
  - rosser 8y ago
    > the willingness of regulators to enforce existing rules, or people being willing and able to sue in court... Regulatory capture and mandatory arbitration agreements buried in the click-wrap. Our "corporate overlords" have structured those factors to work in their favor.
ConceptJunkie 8y ago
So, just like the government, then.
- vkou 8y ago
  Except that with many governments, you have democratic or procedural recourse (When some low level clerk at the DMV arbitrarily cancels your driver's license, you can almost certainly escalate, demand more transparency, etc. When some low level clerk at Amazon closes your account, you are screwed. Appeals will give you no transparency.) With business, your only recourse is the equivalent of moving to a different country. It's the perfect storm of bureaucracy, the-computer-said-it-so-it-must-be-true, for-profit motives, and lack of transparency.
  - ConceptJunkie 8y ago
    Yes, when the government is acting as intended, this is the case, but it's far too easy for evil people to use government power to do evil things. And the more powerful we make government, the more this happens. And in those cases, you seldom have recourse.
savanaly 8y ago
If a company routinely closes its customers' accounts when there's no good reason to won't it open a space for a competitor who is known for not doing that? Seems like it's hard to have an equilibrium where one can't keep a bank account open and no bank is trying to grab more market share by reforming its system to prevent unnecessary closures.
- loceng 8y ago
  Maybe. Google has economies of scale on its side already and automation allows for higher profit, so so long as it isn't regulated it's practically and nearly impossible, an uphill battle.
- ribosometronome 8y ago
  How often does a large company really get brought down by screwing over some small subset of their customers?
  - savanaly 8y ago
    Guess I was more responding to the speculation of a Skynet scenario than of a company screwing over a "small subset" of customers which plainly does routinely happen.
- deleted 8y ago
  [deleted]
- TheOtherHobbes 8y ago
  It's not as if the market can magic gigantic competitive companies into existence. Amazon is a monopoly because of its huge scale. It happens - almost incidentally - to have an effective monopoly on a large sector of the self-publishing market. But that's a small part of a much larger online retail marketplace monopoly. This means it can dictate terms to authors, and generally do whatever the hell it wants to. There is nowhere else that authors can go without losing significant sales. How easy do you think it would be for a competitor to build a monopoly the size of Kindle from a cold start? How much cash and other resources would be required, and how long would it take even if the cash and other resources were available? And after all that - how profitable would that competitor be compared to Amazon, and how likely to provide a reasonable return on IPO?
loceng 8y ago
It sounds like we'll need government agencies, even perhaps tying it into a local police forces' work, to take these issues seriously - as serious as a crime - to make sure answers and reasons are found. Ignorance is bliss and reduces cost for a for-profit business, however it's harmful and negligent and perhaps verges on abusive.
- ams6110 8y ago
  Not a law enforcement problem at all, unless there's some kind of fraud going on. Nobody has a right to an Amazon account. That's totally up to Amazon to manage how they want to.
  - loceng 8y ago
    Perhaps not currently a law enforcement problem - the police are to serve and protect society. We don't let private companies do whatever they want, and this may evolve into a situation - especially if using AI is incentivized by cost benefit - where police can be further utilized.
  - damon_c 8y ago
    Maybe the structure of it will be that we have a right to our data and Amazon is obligated to provide access to it unless a reason is provided.
  - risotto_groupon 8y ago
    Its all fun and games until someone dies in an elevator... https://www.youtube.com/watch?v=sAz_UvnUeuU https://www.youtube.com/watch?v=sAz_UvnUeuU
  - waisbrot 8y ago
    For now, yes. Historically, corporations become more and more abusive until there's a public backlash that gets them regulated. Something like "there must be some clearly-defined way to get a response from a human" seems like plausible legislation.
    - 8y ago
dv_dt 8y ago
So less like 1984 and more like the movie Brazil? It was not too long ago that Amazon used to be known for amazing, human, customer service.
- s17n 8y ago
  Great customer service but ruthless with their suppliers. The author here falls into the latter camp.
  - dv_dt 8y ago
    Good point, though I feel like their customer service has slipped too - e.g. allowing counterfeits to mix into supply of products is not customer friendly.
beamatronic 8y ago
The consequence is that you have to start thinking of your entire digital life as disposable. I hope a company will rise to solve this problem and fill this need.
- softawre 8y ago
  Interesting idea, how does one do this? Make friends locally, save a ton of money (in case you can't work virtually later on), have entertainment locally (backed up or IRL), what else?
  - ryandrake 8y ago
    Yes, yes, and yes. And local backups of all the things. Don’t rely on anything you don’t host yourself, on your own metal. With the exception of a VPS and possibly my AppleID, I could probably lose all my online accounts today and it would not result in meaningful data loss. I’m planning to move my (VPS-hosted) Email on-site one day, and after that’s done my ISP is the last remaining point of failure.
    - Domenic_S 8y ago
      > your own metal It's absolutely maddening that I can't update my video card driver without logging into an nvidia account. That's the direction "your own metal" is going to keep going though.
      - justinclift 8y ago
        If you roll back your driver to an older version of GeForce Experience - one that doesn't require the nvidia account - wouldn't that let you pull down just driver updates? eg as long as you don't update the GeForce Experience program, you should be ok?
  - beamatronic 8y ago
    Exactly, you almost need to be sort of a prepper, with some cash on hand. But I was thinking of the whole vertical stack (backup email address, cell phone, credit cards). I’m thinking “continuity of business” preparations.
vidarh 8y ago
The TV series "Little Britain" had a regular sketch featuring an apathetic bank employee who just checked the computer after every request made by a customer and read out "computer says no" every time, never with any justification to the seemingly completely arbitrary rejections. For banks it reflected the reality at the time (2000-2007), but it's creeping into more and more areas.
- reaperducer 8y ago
  > For banks it reflected the reality at the time (2000-2007) Today, too. Just try to get a mortgage at Chase.
- mmt 8y ago
  In later episodes, the character turned not just apathetic but hostile and sometimes didn't bother even checking the computer before saying "computer says no". As other commenters pointed out elsewhere in the thread, bureaucracy is a way for people/institutions to say "no" and automation is just a modern layer on top of that.
dsnuh 8y ago
I feel like the "rise of the machines" won't be some bloodbath with terminators or AI killing people outright, it will just be that everything will be a Kafka-esque maze of bureaucratic complexity with no human recourse when you get caught in the system. It's like those Coke Freestyle machines, where you get stuck in line behind someone trying to figure out the touchscreen system when all you want to do is get a Coke. The future of society is all of us frustratedly waving our hands in front of an automated paper towel dispenser before we just give up and wipe our hands on our pants.
- danielvf 8y ago
  And yet, it's not technology that does this. It's the inherent nature of systems, and has been around for a long time: https://www.youtube.com/watch?v=JtEkUmYecnk https://www.youtube.com/watch?v=JtEkUmYecnk
  - jandrese 8y ago
    At times it seems the whole point of bureaucracy is to empower people to say "no" even in cases where the complainant has a valid case. You save money by nickles and dimes by forcing people to be a slave to the process and not allow any pesky sense of justice or empathy to impact their moneymaking performance.
- temp-dude-87844 8y ago
  This is already the case. Much of the adult condition consists of trying to prove one's case to bureaucracies: why you shouldn't have incurred that overdraft, why you shouldn't have been denied that insurance claim, why you think there's a mistake on your bill -- and merely getting a human on the phone isn't sufficient to resolve your issue. You should also come armed with ample documentation (which you may not even have) to support your case, and be prepared to set aside a chunk of time as you bounce from department to department. This is part of why money-poor and time-constrained people have such terrible outcomes: each mistake hurts more in proportion, they lack the funds to hire an expert (e.g. tax professional, lawyer), and they're constrained on time to deal with offices that are only open for less than a quarter of the week. There's a march towards more automation, more externally-opaque algorithms, and less humans to escalate to, but the core of the problem is quite old. Instead of a polite-if-flustered customer service representative telling you they don't know, you need to speak to someone else, you'll have to scrape the secret number from a 3-year-old reddit post or publicly shame them on Twitter to get answers, but the outcome's the same: the better-equipped and loudest voices win, while there's a bunch of others who get shafted with no recourse. Eventually, you quietly learn to keep thorough records, emergency savings, and eat the loss, because unless you can fight these sorts of issues in a higher forum where damages are an option, fighting them more than two roundtrips deep isn't worth your time.
trumped 8y ago
This is just the first step, it will get worst.
grahamburger 8y ago
I'm at still irrationally frustrated about my fledgling MailChimp account being flagged and locked months ago for no discernible reason and with no meaningful option for recourse. I am now a happy MailerLite customer and will not use or recommend MailChimp.
- pmiller2 8y ago
  Sounds more like you are rationally frustrated about losing your MailChimp account, to me.

dingaling 8y ago

In the past we spread our patronage and investments around to protect against the failure of businesses. Looks like in the future we'll do so to protect ourselves against AI.

ConceptJunkie 8y ago
Maybe there will be a flipside to this. Perhaps we can hope for the ability to have our own AIs that work on our behalf.
- waisbrot 8y ago
  Maybe if the story author had written a bot to reply "I don't understand. Please tell me what I did wrong." to every bot-generated email from Amazon, he could have made a more entertaining story. He could include some graphs of the reply-counts, estimate the AZ bot's latency, and draw out some maps of the various routes taken by the messages.

kevin_b_er 8y ago

And if you're in the US, there are no restrictions on corporations restricting your rights. Only the government. The more that is privatized, the less rights you have.

Finnucane 8y ago
>And if you're in the US, there are no restrictions on corporations restricting your rights. That's not precisely true, though in practice it may depend on the willingness of regulators to enforce existing rules, or people being willing and able to sue in court. Our corporate overlords can often count on those factors working in their favor.
- rosser 8y ago
  > the willingness of regulators to enforce existing rules, or people being willing and able to sue in court... Regulatory capture and mandatory arbitration agreements buried in the click-wrap. Our "corporate overlords" have structured those factors to work in their favor.

ConceptJunkie 8y ago

So, just like the government, then.

vkou 8y ago
Except that with many governments, you have democratic or procedural recourse (When some low level clerk at the DMV arbitrarily cancels your driver's license, you can almost certainly escalate, demand more transparency, etc. When some low level clerk at Amazon closes your account, you are screwed. Appeals will give you no transparency.) With business, your only recourse is the equivalent of moving to a different country. It's the perfect storm of bureaucracy, the-computer-said-it-so-it-must-be-true, for-profit motives, and lack of transparency.
- ConceptJunkie 8y ago
  Yes, when the government is acting as intended, this is the case, but it's far too easy for evil people to use government power to do evil things. And the more powerful we make government, the more this happens. And in those cases, you seldom have recourse.

savanaly 8y ago

If a company routinely closes its customers' accounts when there's no good reason to won't it open a space for a competitor who is known for not doing that? Seems like it's hard to have an equilibrium where one can't keep a bank account open and no bank is trying to grab more market share by reforming its system to prevent unnecessary closures.

loceng 8y ago
Maybe. Google has economies of scale on its side already and automation allows for higher profit, so so long as it isn't regulated it's practically and nearly impossible, an uphill battle.
ribosometronome 8y ago
How often does a large company really get brought down by screwing over some small subset of their customers?
- savanaly 8y ago
  Guess I was more responding to the speculation of a Skynet scenario than of a company screwing over a "small subset" of customers which plainly does routinely happen.
deleted 8y ago
[deleted]
TheOtherHobbes 8y ago
It's not as if the market can magic gigantic competitive companies into existence. Amazon is a monopoly because of its huge scale. It happens - almost incidentally - to have an effective monopoly on a large sector of the self-publishing market. But that's a small part of a much larger online retail marketplace monopoly. This means it can dictate terms to authors, and generally do whatever the hell it wants to. There is nowhere else that authors can go without losing significant sales. How easy do you think it would be for a competitor to build a monopoly the size of Kindle from a cold start? How much cash and other resources would be required, and how long would it take even if the cash and other resources were available? And after all that - how profitable would that competitor be compared to Amazon, and how likely to provide a reasonable return on IPO?

loceng 8y ago

It sounds like we'll need government agencies, even perhaps tying it into a local police forces' work, to take these issues seriously - as serious as a crime - to make sure answers and reasons are found. Ignorance is bliss and reduces cost for a for-profit business, however it's harmful and negligent and perhaps verges on abusive.

ams6110 8y ago
Not a law enforcement problem at all, unless there's some kind of fraud going on. Nobody has a right to an Amazon account. That's totally up to Amazon to manage how they want to.
- loceng 8y ago
  Perhaps not currently a law enforcement problem - the police are to serve and protect society. We don't let private companies do whatever they want, and this may evolve into a situation - especially if using AI is incentivized by cost benefit - where police can be further utilized.
- damon_c 8y ago
  Maybe the structure of it will be that we have a right to our data and Amazon is obligated to provide access to it unless a reason is provided.
- risotto_groupon 8y ago
  Its all fun and games until someone dies in an elevator... https://www.youtube.com/watch?v=sAz_UvnUeuU https://www.youtube.com/watch?v=sAz_UvnUeuU
- waisbrot 8y ago
  For now, yes. Historically, corporations become more and more abusive until there's a public backlash that gets them regulated. Something like "there must be some clearly-defined way to get a response from a human" seems like plausible legislation.
  - 8y ago

dv_dt 8y ago

So less like 1984 and more like the movie Brazil? It was not too long ago that Amazon used to be known for amazing, human, customer service.

s17n 8y ago
Great customer service but ruthless with their suppliers. The author here falls into the latter camp.
- dv_dt 8y ago
  Good point, though I feel like their customer service has slipped too - e.g. allowing counterfeits to mix into supply of products is not customer friendly.

beamatronic 8y ago

The consequence is that you have to start thinking of your entire digital life as disposable. I hope a company will rise to solve this problem and fill this need.

softawre 8y ago
Interesting idea, how does one do this? Make friends locally, save a ton of money (in case you can't work virtually later on), have entertainment locally (backed up or IRL), what else?
- ryandrake 8y ago
  Yes, yes, and yes. And local backups of all the things. Don’t rely on anything you don’t host yourself, on your own metal. With the exception of a VPS and possibly my AppleID, I could probably lose all my online accounts today and it would not result in meaningful data loss. I’m planning to move my (VPS-hosted) Email on-site one day, and after that’s done my ISP is the last remaining point of failure.
  - Domenic_S 8y ago
    > your own metal It's absolutely maddening that I can't update my video card driver without logging into an nvidia account. That's the direction "your own metal" is going to keep going though.
    - justinclift 8y ago
      If you roll back your driver to an older version of GeForce Experience - one that doesn't require the nvidia account - wouldn't that let you pull down just driver updates? eg as long as you don't update the GeForce Experience program, you should be ok?
- beamatronic 8y ago
  Exactly, you almost need to be sort of a prepper, with some cash on hand. But I was thinking of the whole vertical stack (backup email address, cell phone, credit cards). I’m thinking “continuity of business” preparations.

vidarh 8y ago

The TV series "Little Britain" had a regular sketch featuring an apathetic bank employee who just checked the computer after every request made by a customer and read out "computer says no" every time, never with any justification to the seemingly completely arbitrary rejections. For banks it reflected the reality at the time (2000-2007), but it's creeping into more and more areas.

reaperducer 8y ago
> For banks it reflected the reality at the time (2000-2007) Today, too. Just try to get a mortgage at Chase.
mmt 8y ago
In later episodes, the character turned not just apathetic but hostile and sometimes didn't bother even checking the computer before saying "computer says no". As other commenters pointed out elsewhere in the thread, bureaucracy is a way for people/institutions to say "no" and automation is just a modern layer on top of that.

dsnuh 8y ago

I feel like the "rise of the machines" won't be some bloodbath with terminators or AI killing people outright, it will just be that everything will be a Kafka-esque maze of bureaucratic complexity with no human recourse when you get caught in the system. It's like those Coke Freestyle machines, where you get stuck in line behind someone trying to figure out the touchscreen system when all you want to do is get a Coke. The future of society is all of us frustratedly waving our hands in front of an automated paper towel dispenser before we just give up and wipe our hands on our pants.

danielvf 8y ago
And yet, it's not technology that does this. It's the inherent nature of systems, and has been around for a long time: https://www.youtube.com/watch?v=JtEkUmYecnk https://www.youtube.com/watch?v=JtEkUmYecnk
- jandrese 8y ago
  At times it seems the whole point of bureaucracy is to empower people to say "no" even in cases where the complainant has a valid case. You save money by nickles and dimes by forcing people to be a slave to the process and not allow any pesky sense of justice or empathy to impact their moneymaking performance.
temp-dude-87844 8y ago
This is already the case. Much of the adult condition consists of trying to prove one's case to bureaucracies: why you shouldn't have incurred that overdraft, why you shouldn't have been denied that insurance claim, why you think there's a mistake on your bill -- and merely getting a human on the phone isn't sufficient to resolve your issue. You should also come armed with ample documentation (which you may not even have) to support your case, and be prepared to set aside a chunk of time as you bounce from department to department. This is part of why money-poor and time-constrained people have such terrible outcomes: each mistake hurts more in proportion, they lack the funds to hire an expert (e.g. tax professional, lawyer), and they're constrained on time to deal with offices that are only open for less than a quarter of the week. There's a march towards more automation, more externally-opaque algorithms, and less humans to escalate to, but the core of the problem is quite old. Instead of a polite-if-flustered customer service representative telling you they don't know, you need to speak to someone else, you'll have to scrape the secret number from a 3-year-old reddit post or publicly shame them on Twitter to get answers, but the outcome's the same: the better-equipped and loudest voices win, while there's a bunch of others who get shafted with no recourse. Eventually, you quietly learn to keep thorough records, emergency savings, and eat the loss, because unless you can fight these sorts of issues in a higher forum where damages are an option, fighting them more than two roundtrips deep isn't worth your time.

trumped 8y ago

This is just the first step, it will get worst.

grahamburger 8y ago

I'm at still irrationally frustrated about my fledgling MailChimp account being flagged and locked months ago for no discernible reason and with no meaningful option for recourse. I am now a happy MailerLite customer and will not use or recommend MailChimp.

pmiller2 8y ago
Sounds more like you are rationally frustrated about losing your MailChimp account, to me.