3 ms·

So first of all this isn't a new thing. The statistical methods vs human judgement debate goes back decades. Long before computers, people were training simple

So first of all this isn't a new thing. The statistical methods vs human judgement debate goes back decades. Long before computers, people were training simple linear models with pencil and paper. The earliest paper I found is from 1928, on a similar issue as in the article. Where a very crude statistical algorithm was better at predicting recidivism of inmates, than three prison psychologists. This isn't a new thing at all. Second the models in question aren't as complicated as you imagine. These are just decision trees with a few hundred inputs at most. Even with more complicated models, you can train simpler models to mimic them, and then inspect at those. There are other ways to make algorithms more transparent, but my point is these aren't complicated image recognition deep neural nets. They don't need to be either. Third I don't think these algorithms are "dumb". By all accounts they significantly outperform humans. Humans, even experts, are terrible at doing even basic statistics in their head. Often our decisions aren't much better than random chance. Even when humans are allowed to see the results of an algorithm, and tweak it's output when they think it's making a mistake, they do worse than just the algorithm alone. There's now evidence to suggest humans are irrationally biased against algorithms. Search for "algorithm aversion". A study shows that even after watching a statistical algorithm do better, people still prefer worse human judgement.

dmreedy 10y ago
Yep, like I said, it's absolutely an old problem, even when it comes to statistical techniques. I understand that the particular models in the article are on the simpler side. I was speaking more generally. As to whether the models in question here need to be more complicated, well, that is an interesting question. Humans are absolutely terrible at doing basic statistics in their head. Algorithm aversion is probably a thing, no doubt motivated by similar anthrocentric biases to luddism. My point was there are other ways of measuring how 'good' a system is. In the space of a well-defined problem with well-defined parameters, sure, maybe accuracy is king. This is strikes me as akin to trying to build a bridge while assuming that the world is a frictionless vacuum. If everyone has agreed that the problem is correctly framed, then yes, a statistical system can be trusted, the mathematics are inevitable. But a statistical model (the current iterations of them, at least) is never going to ask whether or not it -should- only be considering the inputs its given, or whether more is needed. The world that these systems understand consists of the subset of the world provided to them by a set of sample data The patterns they learn can only ever be as good as that. That's what I mean by 'dumb'. Not that they aren't good at what they do. That they aren't good at knowing about what they can't do.

dmreedy 10y ago

Yep, like I said, it's absolutely an old problem, even when it comes to statistical techniques. I understand that the particular models in the article are on the simpler side. I was speaking more generally. As to whether the models in question here need to be more complicated, well, that is an interesting question. Humans are absolutely terrible at doing basic statistics in their head. Algorithm aversion is probably a thing, no doubt motivated by similar anthrocentric biases to luddism. My point was there are other ways of measuring how 'good' a system is. In the space of a well-defined problem with well-defined parameters, sure, maybe accuracy is king. This is strikes me as akin to trying to build a bridge while assuming that the world is a frictionless vacuum. If everyone has agreed that the problem is correctly framed, then yes, a statistical system can be trusted, the mathematics are inevitable. But a statistical model (the current iterations of them, at least) is never going to ask whether or not it -should- only be considering the inputs its given, or whether more is needed. The world that these systems understand consists of the subset of the world provided to them by a set of sample data The patterns they learn can only ever be as good as that. That's what I mean by 'dumb'. Not that they aren't good at what they do. That they aren't good at knowing about what they can't do.