3 ms·

I know what you are saying but I want to counter the interpretation a bit. What the spokespeople say is a claim…it’s not truth, we don’t have independent evide

by avs733 3y ago

I know what you are saying but I want to counter the interpretation a bit. What the spokespeople say is a claim…it’s not truth, we don’t have independent evidence of it, of course they would claim that. Even if they did have human in the loop - what does that meaningfully look like? Are those humans given the authority to actually overrule the decision? Are they given the evidence to do so? Are they punished or perceive that they will be punished if they do? I don’t think taking the banks answer to any of those questions at face value is responsible citizenship. Yes it’s what they said - no I don’t think that supports calling it reality. I’ve mentioned this in other threads related to medical billing. My wife and I have had crazy medical bills for bizarre things the last few years. They are never correct, often they are basically fiction. The people you talk to just point to the bill as if it is inherently correct because it’s the bill. I’ve shown pictures of me on vacation on another continent that same day they charged me for a service…well the bills the bill. That’s not human in the loop, it’s a human shield around the automated decision making loop. It’s a fixable problem, make incorrect billing an actual crime. It already is to mid bill the government, just not people.

hfx21 3y ago
You are right and I am speaking as someone who has worked on these systems. It's hard to convey to people, as these systems scale up, the bugs scale up too. And the resources to handle them shrink. Even if the false positives are less than 1% tens of thousands of people can get effected. And things are swept under the carpet, cause the costs to corporations are rounding errors.
freddie_mercury 3y ago
> I don’t think taking the banks answer to any of those questions at face value is responsible citizenship I agree but I wanted to push back on the idea that this was somehow due to automated decisions being made. That isn't really the problem here. The same exact thing would happen if the government mandated "no computers are allowed to be involved". The problem is opaque decision making, no clear escalation path, a disregard for the short term pains, and so on. The fundamental problem isn't how the decision gets made. Even if a human made the decision you'd have the same issues of the front line consumer support saying "well the other department made their decision, I don't know why and I don't have the authority to change it and they don't accept calls". By talking about automated decisions, the OP is making the conversation about the wrong problem.
- avs733 3y ago
  I can agree with this. I do think automated decision making amplifies the effect, but you are right on the fundamental issue

by avs733 3y ago

hfx21 3y ago

You are right and I am speaking as someone who has worked on these systems. It's hard to convey to people, as these systems scale up, the bugs scale up too. And the resources to handle them shrink. Even if the false positives are less than 1% tens of thousands of people can get effected. And things are swept under the carpet, cause the costs to corporations are rounding errors.

freddie_mercury 3y ago

> I don’t think taking the banks answer to any of those questions at face value is responsible citizenship I agree but I wanted to push back on the idea that this was somehow due to automated decisions being made. That isn't really the problem here. The same exact thing would happen if the government mandated "no computers are allowed to be involved". The problem is opaque decision making, no clear escalation path, a disregard for the short term pains, and so on. The fundamental problem isn't how the decision gets made. Even if a human made the decision you'd have the same issues of the front line consumer support saying "well the other department made their decision, I don't know why and I don't have the authority to change it and they don't accept calls". By talking about automated decisions, the OP is making the conversation about the wrong problem.

avs733 3y ago
I can agree with this. I do think automated decision making amplifies the effect, but you are right on the fundamental issue