4 ms·

I just saw another article[1] on a study using the same data set. It concluded that the COMPAS algorithm is no more accurate than random people given the task o

I just saw another article[1] on a study using the same data set. It concluded that the COMPAS algorithm is no more accurate than random people given the task on Mechanical Turk. Those researchers also designed a simple predictive model using only two factors as inputs: age and number of previous convictions. By comparison, COMPAS uses more than a hundred. To me, it doesn't sound like those extra factors end up contributing much. [1]: https://www.economist.com/news/science-and-technology/21734986-short-answer-two-are-about-same-are-programs-better https://www.economist.com/news/science-and-technology/217349...

noobermin 9y ago
Off topic, I can't access the economist article, could you post a link to the study so I and/or others can read its abstract?
- indubitable 9y ago
  http://advances.sciencemag.org/content/4/1/eaao5580.full http://advances.sciencemag.org/content/4/1/eaao5580.full Not paywalled.
AnthonyMouse 9y ago
> Those researchers also designed a simple predictive model using only two factors as inputs: age and number of previous convictions. By comparison, COMPAS uses more than a hundred. To me, it doesn't sound like those extra factors end up contributing much. That's the nature of these things. If you use zero factors then you're basing decisions on random chance. Use one good factor and you'll do 60% better than random chance. Add another good factor and you'll do another 10% better. Every factor after that is a single digit improvement. Because once you have something pretty good, making it even better is diminishing returns. But at the same time, "it's only 1% more accurate that way" when you're dealing with country-sized populations will change the outcome for many thousands of people. And how does using fewer factors help anything? The underlying "problem" is that people of different races have different outcomes in actual fact, and any halfway accurate predictive algorithm will reflect that difference. It's not as if there is no racial disparity in the number of previous convictions. If anything, using fewer factors could make the problem worse because it stops considering potential mitigating factors. A young black man with two prior convictions gets the shaft even if he has recently gotten married and bought a house in a better neighborhood.
indubitable 9y ago
That is a very poorly formed study precisely because systems like the Mechanical Turk actually work surprisingly well. This is a common theme in the 'wisdom of the masses.' If you ask a single person how many beans are in a jar, you're going to get an answer that's generally very wrong. Yet ask 100 people and average their answer and you tend to get an answer that's oddly extremely close to the correct answer. As the number of people independently asked approaches infinity, the error approaches 0. So when you ask 'x' people to independently judge something, let alone something that is multiple choice with a correct answer, you're going to get answers that are far more accurate than a single individual would give you. So looking at the average answer of 'x' people, comparing it to an AI system, and arguing that they're relatively close so individual people are relatively close to the AI is completely fallacious. --- As a tangential aside, there's another quirk to the wisdom of the masses. When you let the people communicate and try to intelligently organize and use expertise to come to answer, this effect disappears and the final answer again tends to be very wrong. Kind of an interesting perspective on the current zeitgeist of society and work. ===EDIT=== The authors were obviously aware of the wisdom of the masses. Quoting the paper itself: To determine whether there is “wisdom in the crowd” (7) (in our case, a small crowd of 20 per subset), participant responses were pooled within each subset using a majority rules criterion. This crowd-based approach yields a prediction accuracy of 67.0%. A one-sided t test reveals that COMPAS is not significantly better than the crowd (P = 0.85). That's a quite silly p-value and on top of that I'm not sure how they claim their system actually controls for the wisdom of the masses.
- IshKebab 9y ago
  > As the number of people independently asked approaches infinity, the error approaches 0. Only if people are an unbiased estimator!!
  - indubitable 9y ago
    You'd think so, but that's not correct. This is not a straight forward result of probability with a filter of complexity. It works regardless of bias, though obviously if everybody was biased in the exact same way then that would cause things to break down - which is perhaps the reason that the coordination results in a worse result than independent averages. For another example of it consider things like the television show, 'Who wants to be a millionaire?' It's a quiz show where one of the choices is for the participant to ask the audience. And the audience tends to do absurdly well on even the most esoteric questions, though independently they are certainly far from trivia experts. But very few are randomly guessing - their own experiences and biases leads them to entirely different conclusions. Yet somehow it produces the correct result time and again. It's a strange phenomena and one that has to be constantly guarded against in anything involving sampling of people. This is a text book example of a study that gets destroyed by it.

noobermin 9y ago

Off topic, I can't access the economist article, could you post a link to the study so I and/or others can read its abstract?

indubitable 9y ago
http://advances.sciencemag.org/content/4/1/eaao5580.full http://advances.sciencemag.org/content/4/1/eaao5580.full Not paywalled.

AnthonyMouse 9y ago

> Those researchers also designed a simple predictive model using only two factors as inputs: age and number of previous convictions. By comparison, COMPAS uses more than a hundred. To me, it doesn't sound like those extra factors end up contributing much. That's the nature of these things. If you use zero factors then you're basing decisions on random chance. Use one good factor and you'll do 60% better than random chance. Add another good factor and you'll do another 10% better. Every factor after that is a single digit improvement. Because once you have something pretty good, making it even better is diminishing returns. But at the same time, "it's only 1% more accurate that way" when you're dealing with country-sized populations will change the outcome for many thousands of people. And how does using fewer factors help anything? The underlying "problem" is that people of different races have different outcomes in actual fact, and any halfway accurate predictive algorithm will reflect that difference. It's not as if there is no racial disparity in the number of previous convictions. If anything, using fewer factors could make the problem worse because it stops considering potential mitigating factors. A young black man with two prior convictions gets the shaft even if he has recently gotten married and bought a house in a better neighborhood.

indubitable 9y ago

That is a very poorly formed study precisely because systems like the Mechanical Turk actually work surprisingly well. This is a common theme in the 'wisdom of the masses.' If you ask a single person how many beans are in a jar, you're going to get an answer that's generally very wrong. Yet ask 100 people and average their answer and you tend to get an answer that's oddly extremely close to the correct answer. As the number of people independently asked approaches infinity, the error approaches 0. So when you ask 'x' people to independently judge something, let alone something that is multiple choice with a correct answer, you're going to get answers that are far more accurate than a single individual would give you. So looking at the average answer of 'x' people, comparing it to an AI system, and arguing that they're relatively close so individual people are relatively close to the AI is completely fallacious. --- As a tangential aside, there's another quirk to the wisdom of the masses. When you let the people communicate and try to intelligently organize and use expertise to come to answer, this effect disappears and the final answer again tends to be very wrong. Kind of an interesting perspective on the current zeitgeist of society and work. ===EDIT=== The authors were obviously aware of the wisdom of the masses. Quoting the paper itself: To determine whether there is “wisdom in the crowd” (7) (in our case, a small crowd of 20 per subset), participant responses were pooled within each subset using a majority rules criterion. This crowd-based approach yields a prediction accuracy of 67.0%. A one-sided t test reveals that COMPAS is not significantly better than the crowd (P = 0.85). That's a quite silly p-value and on top of that I'm not sure how they claim their system actually controls for the wisdom of the masses.

IshKebab 9y ago
> As the number of people independently asked approaches infinity, the error approaches 0. Only if people are an unbiased estimator!!
- indubitable 9y ago
  You'd think so, but that's not correct. This is not a straight forward result of probability with a filter of complexity. It works regardless of bias, though obviously if everybody was biased in the exact same way then that would cause things to break down - which is perhaps the reason that the coordination results in a worse result than independent averages. For another example of it consider things like the television show, 'Who wants to be a millionaire?' It's a quiz show where one of the choices is for the participant to ask the audience. And the audience tends to do absurdly well on even the most esoteric questions, though independently they are certainly far from trivia experts. But very few are randomly guessing - their own experiences and biases leads them to entirely different conclusions. Yet somehow it produces the correct result time and again. It's a strange phenomena and one that has to be constantly guarded against in anything involving sampling of people. This is a text book example of a study that gets destroyed by it.