3 ms·
This is a great summary, and that Citylab piece is well worth checking out. I'm sometimes skeptical of human-narrative analyses, because they make it easy to l
by Bartweiss 8y ago
This is a great summary, and that Citylab piece is well worth checking out.
I'm sometimes skeptical of human-narrative analyses, because they make it easy to launch equally-vivid attacks on a plan that fails 10 people and a plan that fails 10,000. But with rigor behind them, there's a lot to be said for them as existence proofs - especially when we're talking about the margins of a plan instead of its core. Knowing what the edge cases look like matters, and for that it's hard to beat talking to individuals with problems that don't fit on a form.
(As an aside, I think software often creates not only an empathy gap but a mechanical one. An analog system can often be forced to accommodate oddities like self-employment and unconventional address formats, but "the computer won't let me enter that address" is a brick wall. I've never found an investigation of this, but I suspect it's a major drawback of digitization.)
Your final point about ML is a vital one, and one I'm endlessly frustrated by. We're increasingly aware of issues like racial bias in parole-predictions systems, but there's alarmingly little focus on the fact that a system which does that is flawed in general, and might be making the same assumptions about e.g. low-income neighborhoods without anyone noticing. Unlike human bias, a flawed algorithm is equally happy to abuse any strange correlation it can find.