4 ms·

Why is it terrifying? It's accurate. A bug plus shitty automated support with no human in the loop is an intentional product decision. They could have humans i

Why is it terrifying? It's accurate. A bug plus shitty automated support with no human in the loop is an intentional product decision. They could have humans in the loop. They could have human support. They could make bugs like this more easily remediable. But they don't. On purpose. To the extent that their "fare review process" and the app failed. Now it's going to be fixed by some PR person. Because it's still cheaper, apparently, than the alternative design decisions. This outcome is a result of their intentional product decisions.

derefr 5y ago
Yes, but that's still a different thing than "a human sat down and decided that there would be trips that would cost the driver money to run; and that Uber wouldn't announce this change." One is an emergent consequence — the "banality of evil", a property of the corporation itself. The other is active malfeasance — something that's not so much even about the corporation as about an individual horrible person in a decision-making role, where the correct move for customers/employees is to boycott the decision-maker, rather than the company; and where the correct move for the company is to fire the decision-maker, and distance themselves from them as much as possible. (Unless it keeps happening with different people in the same company, at which point it becomes a thing about the company's culture.)
- Analemma_ 5y ago
  Businesses deliberately plan for and rely on people like you making that distinction, even though you shouldn’t. Why should it matter? The consumer is fucked either way.
  - derefr 5y ago
    For the same reason that if some kind of Cordyceps-like brain-parasite were to infect a person and make them murder people, that wouldn't be that person's fault. Especially if the brain parasite could be removed, reverting their personality to normal. I wouldn't hold that against the person going forward (unless they for some reason actively pursued further brain-parasite infections as a general preference.) Companies can be evil on a structural level. But also, companies that aren't evil on a structural level, can be parasitized by evil individuals. Unless the company actively pursues hiring evil individuals, why would you hold the latter against them? The point of companies having reputations is to predict what they'll do, and thereby trade with companies for which trade with them will predictably be positive-sum, and avoid trade with companies that would predictably be zero-sum or negative-sum. A company itself being evil on a structural level, predicts that in the future it will do many zero- or negative-sum things, regardless of employee turnover. A company that is not evil on a structural level, hiring one evil individual, who does an evil thing and then gets fired, doesn't predict any future evil behavior from the company.
    - Kinrany 5y ago
      You're missing the actually relevant case: the company hires evil individuals on a regular basis.
    - buu700 5y ago
      That's an interesting way of framing the issue. By that logic, while active malfeasance may be the scenario that generates more outrage, it's actually the banal/emergent evil that represents more of a reason to avoid a business because between the two it's much harder to fix.
    - praestigiare 5y ago
      I don't know where the talk about evil individuals came from - Uber's billing is not the result of a single "potentially evil" individual. Instead it is the result of the company culture and processes. This incident was not an intentional decision to charge drivers to work, but still a predictable outcome of the way they operate, and thus indistinguishable from actual malice from the perspective of the customer, both in terms of result and predictive value.
      - derefr 5y ago
        > I don't know where the talk about evil individuals came from - Uber's billing is not the result of a single "potentially evil" individual. It's what would be true for the situation outlined in the root comment of this subthread to pertain: that it was "an intentional product decision." For that to be true, there would need to be an individual that made that decision, with the thought-process I outlined in my first comment. Such a thought-process wouldn't just be sociopathic (zero-sum), it'd be negative-sum/"evil". It would be a choice to hurt both drivers, and the corporation itself (when people inevitably find out), and oneself (when blame is inevitably pointed at them.) For what? Who knows. Evil. (Industrial sabotage, maybe.) Obviously, like another commenter said, this wasn't an intentional product decision. (It was maybe an intentional project management decision, but one made in by no one individual.) But that same commenter rhetorically asked why it mattered whether it was an intentional product decision or not. That's why I explained the difference. Acts of evil individuals that have parasitized good companies shouldn't reflect on those companies (once the evil individual is removed.) Acts of structural evil — acts that are generated from an algorithm the corporation is following as a whole, and would still follow if you replaced every employee ship-of-Theseus style — should reflect on the company itself. This, of course, is an example of the latter kind of act, one that should reflect on the company itself. But it's important to distinguish these acts from acts that are, in fact, committed by bad actors that have "infiltrated" the company. For the same reason that we distinguish "terrorism" from "domestic terrorism." The latter — domestic terrorism — can, in some ways, be "a reflection of the culture of the country." When there's a school shooting, we can ask what the country is doing wrong that leads to people like this cropping up. But the former — plain old terrorism — is not, and cannot, reflect on the country. The person who committed the act wasn't even an enculturated citizen of the country, so what would it tell us about the country?
pixl97 5y ago
I believe the term we are looking for is 'terrifying accurate'. It won't be long before we have a daily post here on HN called Killed by Algorithms, and every story will be different.
- fnord77 5y ago
  I'll start it off https://en.wikipedia.org/wiki/Therac-25 https://en.wikipedia.org/wiki/Therac-25
strken 5y ago
I'm pretty sure that at no point did anyone sit down and say "let's charge drivers for trips". It's clearly the unintended consequence of real product decisions, but it's equally clearly not a new strategic direction to bill the supply side of the market. It's terrifying because it illustrates the lack of trust that the general public has in tech. Instead of "looks like someone's having a bad day at the Uber office, guess they'll fix it soon," it's a plot trialling a new form of exploitation. Two replies in, someone wants to nationalise the company. That's a lot of lost trust.
- zanellato19 5y ago
  Yeah, but the ramifications of the unending automation is indeed exactly the impossible-to-deal-with bugs. The fact of the matter is that Uber (the company, not any specific PMs/Engineers inside it) decided that this was unnaceptable loss of money and they instead shift that to the customers. That decision, to not "invest" in support, is completely conscious by the founders/investors to make the business more viable, because at their scale, support can be quite expensive. In other words, technology is bringing scale and leaving the cost of the rounding errors to the most vulnerable.
- 3np 5y ago
  They had many years of time to address this class of issues, and sometimes it has consistently gotten worse, not better. Why would the trust not go down?
  - strken 5y ago
    I don't know, why would the trust not go down? Is this a riddle? Of course trust has gone down. I'm not saying it's unexpected, I'm remarking on adverse market conditions that will affect most of HN.
- SilasX 5y ago
  No, but they did say, "let's take a fixed fraction of total charges" rather then "let's take a fixed fraction of the amount after paying third parties", not realizing the former could be more than the driver's share.

derefr 5y ago

Yes, but that's still a different thing than "a human sat down and decided that there would be trips that would cost the driver money to run; and that Uber wouldn't announce this change." One is an emergent consequence — the "banality of evil", a property of the corporation itself. The other is active malfeasance — something that's not so much even about the corporation as about an individual horrible person in a decision-making role, where the correct move for customers/employees is to boycott the decision-maker, rather than the company; and where the correct move for the company is to fire the decision-maker, and distance themselves from them as much as possible. (Unless it keeps happening with different people in the same company, at which point it becomes a thing about the company's culture.)

Analemma_ 5y ago
Businesses deliberately plan for and rely on people like you making that distinction, even though you shouldn’t. Why should it matter? The consumer is fucked either way.
- derefr 5y ago
  For the same reason that if some kind of Cordyceps-like brain-parasite were to infect a person and make them murder people, that wouldn't be that person's fault. Especially if the brain parasite could be removed, reverting their personality to normal. I wouldn't hold that against the person going forward (unless they for some reason actively pursued further brain-parasite infections as a general preference.) Companies can be evil on a structural level. But also, companies that aren't evil on a structural level, can be parasitized by evil individuals. Unless the company actively pursues hiring evil individuals, why would you hold the latter against them? The point of companies having reputations is to predict what they'll do, and thereby trade with companies for which trade with them will predictably be positive-sum, and avoid trade with companies that would predictably be zero-sum or negative-sum. A company itself being evil on a structural level, predicts that in the future it will do many zero- or negative-sum things, regardless of employee turnover. A company that is not evil on a structural level, hiring one evil individual, who does an evil thing and then gets fired, doesn't predict any future evil behavior from the company.
  - Kinrany 5y ago
    You're missing the actually relevant case: the company hires evil individuals on a regular basis.
  - buu700 5y ago
    That's an interesting way of framing the issue. By that logic, while active malfeasance may be the scenario that generates more outrage, it's actually the banal/emergent evil that represents more of a reason to avoid a business because between the two it's much harder to fix.
  - praestigiare 5y ago
    I don't know where the talk about evil individuals came from - Uber's billing is not the result of a single "potentially evil" individual. Instead it is the result of the company culture and processes. This incident was not an intentional decision to charge drivers to work, but still a predictable outcome of the way they operate, and thus indistinguishable from actual malice from the perspective of the customer, both in terms of result and predictive value.
    - derefr 5y ago
      > I don't know where the talk about evil individuals came from - Uber's billing is not the result of a single "potentially evil" individual. It's what would be true for the situation outlined in the root comment of this subthread to pertain: that it was "an intentional product decision." For that to be true, there would need to be an individual that made that decision, with the thought-process I outlined in my first comment. Such a thought-process wouldn't just be sociopathic (zero-sum), it'd be negative-sum/"evil". It would be a choice to hurt both drivers, and the corporation itself (when people inevitably find out), and oneself (when blame is inevitably pointed at them.) For what? Who knows. Evil. (Industrial sabotage, maybe.) Obviously, like another commenter said, this wasn't an intentional product decision. (It was maybe an intentional project management decision, but one made in by no one individual.) But that same commenter rhetorically asked why it mattered whether it was an intentional product decision or not. That's why I explained the difference. Acts of evil individuals that have parasitized good companies shouldn't reflect on those companies (once the evil individual is removed.) Acts of structural evil — acts that are generated from an algorithm the corporation is following as a whole, and would still follow if you replaced every employee ship-of-Theseus style — should reflect on the company itself. This, of course, is an example of the latter kind of act, one that should reflect on the company itself. But it's important to distinguish these acts from acts that are, in fact, committed by bad actors that have "infiltrated" the company. For the same reason that we distinguish "terrorism" from "domestic terrorism." The latter — domestic terrorism — can, in some ways, be "a reflection of the culture of the country." When there's a school shooting, we can ask what the country is doing wrong that leads to people like this cropping up. But the former — plain old terrorism — is not, and cannot, reflect on the country. The person who committed the act wasn't even an enculturated citizen of the country, so what would it tell us about the country?

pixl97 5y ago

I believe the term we are looking for is 'terrifying accurate'. It won't be long before we have a daily post here on HN called Killed by Algorithms, and every story will be different.

fnord77 5y ago
I'll start it off https://en.wikipedia.org/wiki/Therac-25 https://en.wikipedia.org/wiki/Therac-25

strken 5y ago

I'm pretty sure that at no point did anyone sit down and say "let's charge drivers for trips". It's clearly the unintended consequence of real product decisions, but it's equally clearly not a new strategic direction to bill the supply side of the market. It's terrifying because it illustrates the lack of trust that the general public has in tech. Instead of "looks like someone's having a bad day at the Uber office, guess they'll fix it soon," it's a plot trialling a new form of exploitation. Two replies in, someone wants to nationalise the company. That's a lot of lost trust.

zanellato19 5y ago
Yeah, but the ramifications of the unending automation is indeed exactly the impossible-to-deal-with bugs. The fact of the matter is that Uber (the company, not any specific PMs/Engineers inside it) decided that this was unnaceptable loss of money and they instead shift that to the customers. That decision, to not "invest" in support, is completely conscious by the founders/investors to make the business more viable, because at their scale, support can be quite expensive. In other words, technology is bringing scale and leaving the cost of the rounding errors to the most vulnerable.
3np 5y ago
They had many years of time to address this class of issues, and sometimes it has consistently gotten worse, not better. Why would the trust not go down?
- strken 5y ago
  I don't know, why would the trust not go down? Is this a riddle? Of course trust has gone down. I'm not saying it's unexpected, I'm remarking on adverse market conditions that will affect most of HN.
SilasX 5y ago
No, but they did say, "let's take a fixed fraction of total charges" rather then "let's take a fixed fraction of the amount after paying third parties", not realizing the former could be more than the driver's share.