3 ms·

Does it not concern you in the slightest, that when they've ended up speaking to higher and higher up people in the various organisations that they literally do

Does it not concern you in the slightest, that when they've ended up speaking to higher and higher up people in the various organisations that they literally don't know why the algorithm decided the way it did? No one they've spoken to can figure it out, which means they themselves can't also rule out discrimination happening. A lot of the technology we produce is fundamentally biased by our own perceptions, and in ML case most especially by the input data. The danger is a self-perpetuation of already established systemic bias. Given access to good credit is key towards economic mobility that can have massive ongoing implications. The fact that they're saying "It's just what the algorithm produces" should be triggering huge alarm bells, regardless of the veracity of any of his other claims.

deleted 7y ago
[deleted]
xienze 7y ago
> when they've ended up speaking to higher and higher up people in the various organisations that they literally don't know why the algorithm decided the way it did? I don’t think it’s really fair to expect a C-level exec to know the low-level details of how creditworthiness is determined. I’m sure they can paint in broad strokes, but expecting a detailed answer for why a specific person got approved for a certain amount is a bit of a reach. In the case of Woz having a higher limit than his wife, I’m sure it boils down to the usual stuff like, oh I don’t know, not being a cofounder of a massively successful corporation and therefore being filthy rich for the last 30+ years. There might very well be a case to be made here but not likely between two people who are so very different in financial terms (or were for most of their lives). Your credit score isn’t something that changes the instant you get married to someone who’s rich.
powvans 7y ago
Perhaps the people who are "higher and higher up" don't know what the inputs to the algorithm were and therefore can't say one way or the other. Perhaps those inputs are sensitive PII which shouldn't be accessible to the customer support reps who are trying to answer these concerns. Now that's probably cause for concern too, of course. Imagine if you actually saw the massive personal data trove that has been collected on you which is being used to make these decisions. Sends a shiver up the spine doesn't it?
protomyth 7y ago
Does it not concern you in the slightest, that when they've ended up speaking to higher and higher up people in the various organisations that they literally don't know why the algorithm decided the way it did? That isn't what concerns me. I wouldn't expect the CEO of Apple to explain the inner workings of any of their algorithms. That's not their function. What scares me is that people who program or specify these things don't know how the heck decisions are made. From credit decisions to self-driving cars, we are trusting black boxes with no idea why they make specific decisions.
MaupitiBlue 7y ago
> Does it not concern you in the slightest, that when they've ended up speaking to higher and higher up people in the various organisations that they literally don't know why the algorithm decided the way it did? No, I wouldn’t be concerned in the slightest - I’d just get a card from somewhere else.
- MyHypatia 7y ago
  Artificially limiting people's access to capital is a real problem. Sure, perhaps having to go to a separate company once is a minor inconvenience. If no one questions the "algorithm" and it gains wider spread adoption, people's access to capital can very quickly deteriorate. This has real consequences to people's ability to start a business, buy a house, pay a hospital bill... And yet you are not "concerned in the slightest". Do you have a blind trust in the algorithm, or just complete apathy on how this could affect people? What are the things that concern you then?

deleted 7y ago

[deleted]

xienze 7y ago

> when they've ended up speaking to higher and higher up people in the various organisations that they literally don't know why the algorithm decided the way it did? I don’t think it’s really fair to expect a C-level exec to know the low-level details of how creditworthiness is determined. I’m sure they can paint in broad strokes, but expecting a detailed answer for why a specific person got approved for a certain amount is a bit of a reach. In the case of Woz having a higher limit than his wife, I’m sure it boils down to the usual stuff like, oh I don’t know, not being a cofounder of a massively successful corporation and therefore being filthy rich for the last 30+ years. There might very well be a case to be made here but not likely between two people who are so very different in financial terms (or were for most of their lives). Your credit score isn’t something that changes the instant you get married to someone who’s rich.

powvans 7y ago

Perhaps the people who are "higher and higher up" don't know what the inputs to the algorithm were and therefore can't say one way or the other. Perhaps those inputs are sensitive PII which shouldn't be accessible to the customer support reps who are trying to answer these concerns. Now that's probably cause for concern too, of course. Imagine if you actually saw the massive personal data trove that has been collected on you which is being used to make these decisions. Sends a shiver up the spine doesn't it?

protomyth 7y ago

Does it not concern you in the slightest, that when they've ended up speaking to higher and higher up people in the various organisations that they literally don't know why the algorithm decided the way it did? That isn't what concerns me. I wouldn't expect the CEO of Apple to explain the inner workings of any of their algorithms. That's not their function. What scares me is that people who program or specify these things don't know how the heck decisions are made. From credit decisions to self-driving cars, we are trusting black boxes with no idea why they make specific decisions.

MaupitiBlue 7y ago

> Does it not concern you in the slightest, that when they've ended up speaking to higher and higher up people in the various organisations that they literally don't know why the algorithm decided the way it did? No, I wouldn’t be concerned in the slightest - I’d just get a card from somewhere else.

MyHypatia 7y ago
Artificially limiting people's access to capital is a real problem. Sure, perhaps having to go to a separate company once is a minor inconvenience. If no one questions the "algorithm" and it gains wider spread adoption, people's access to capital can very quickly deteriorate. This has real consequences to people's ability to start a business, buy a house, pay a hospital bill... And yet you are not "concerned in the slightest". Do you have a blind trust in the algorithm, or just complete apathy on how this could affect people? What are the things that concern you then?