4 ms·
Let's assume that there is some valid reason for the difference in credit limits. (I'm not assuming one way or the other, but let's just grant it here.) This i
by function_seven 7y ago
Let's assume that there is some valid reason for the difference in credit limits. (I'm not assuming one way or the other, but let's just grant it here.)
This is still bad for Apple and for consumers. The "ALGORITHM" that can't be questioned, inspected, explained, or overruled is a massive failure. Whether its criminal justice, credit scores, or behavioral predictions, ceding authority to some "AI" overlord can't end in just or fair outcomes. (Scare quotes around "AI", because the black box may just be a chain of if/then statements that conveniently proxy in the worst of our institutional biases. Or it could be sophisticated ML... proxying those same things.)
I don't think we're done hearing about this. I'll be very interested in what Apple has to say about it, or what—if anything—is discovered. And it'll doubleplusungood if gender is an explicit input into the "ALGORITHM".
DHH is absolutely right to push back against all the respondents that offer plausible explanations. They're all missing the point. The point is that Apple should be providing a concrete explanation for their credit decisions. All credit providers should.
- matthewmacleod 7y agoYep. This is part of why GDPR is so important - the “right to explanation” for automated decision-making processes.