4 ms·

I am not a fan of this article. I have seen many critiques in this vein, this is just another car in the train. None of them have quite reached the point of c

I am not a fan of this article. I have seen many critiques in this vein, this is just another car in the train. None of them have quite reached the point of confronting what is bothering many: What will we do when machine learning (or science, or anything, really) comes to a conclusion we find unpalatable, for whatever reason? It could be any conclusion, not just those conservatives dislike. Using myself as a target, what if we eventually have enough sampling data to show that people of Irish extraction are more prone to alcoholism, and people of Scottish extraction tend to be statistically more thrifty? (This suggests that I would be a cheap drunk). How will we cope with that? It is true, ML is prone to some black-boxiness, but it could be any statistical extraction. We might very well use other methods besides ML to show the correlates once suggested. Will we simply put in a hard override to get the answers we want to get, the answers we find comfortable? History shows we have seen whole governments subscribe to this idea before. Ignore the results, publish what pleases. I've no easy answers here, but my guess is that history will repeat itself.

moduspol 7y ago
> What will we do when machine learning (or science, or anything, really) comes to a conclusion we find unpalatable, for whatever reason? What makes you think it hasn't already happened? Numerous times? Biological differences between sexes? IQ differences between differentiable subsets of people? The gender equality paradox? We already know what will happen when findings are unpalatable. We'll think of ways to explain why they don't matter and can't possibly be right, or that we weren't measuring the right thing in the first place. And these are just the more obvious cases!
- bsanr2 7y ago
  >or that we weren't measuring the right thing in the first place. Which has been true, actually. For example, human facial recognition systems that can't properly distinguish dark-skinned faces. What is your conclusion in those cases? That dark-skinned people don't have faces? All look the same? Aren't human? Or is it more likely that the system is flawed in some way? Is that flaw in line with existing biases? What then is the chance that the flaw is BECAUSE of our existing biases, in some way or fashion? From what I've read on HN, many people involved in developing ML technology seem to be overly concerned with their systems spitting out "uncomfortable truths", and less concerned with flaws in their data capture, training, or system design processes. We're putting the cart before the horse, and the giddiness with which it's being done is troubling in and of itself.
  - moduspol 7y ago
    Sure--nobody's arguing that we're always measuring the right thing. It may be more relevant in developing ML technology because readers here are the people who have to (at some point) decide whether they've found an uncomfortable truth or falsehood. Or that it's potentially biased, but may or may not be relevant. If I'm doing this at work, I may need to be able to explain to my boss whether or not what we're doing is going to be called racist or sexist, and that's regardless of whether what's being done actually is racist or sexist. It's easy if you're using ML to detect whether an image contains a bird or train. And it's easy when you point to an obvious example with bad data (e.g. inability to differentiate dark faces). But that's not all cases. Sometimes it's good data and the findings are unpalatable. That was the root of the question.
    - bsanr2 7y ago
      We say "unpalatable," but what we really mean is "politically incorrect." There are segments of our society which would be overjoyed to have such "concrete" evidence of their terrible beliefs. Therefore, it is important to observe OP's thesis: that ML "findings" are conservative; that they tend to reproduce preexisting biases; and that we should be interrogating the input data, systems, and output of ML-applied tasks that return "controversial" findings. I agree that it's important to be aware of the potential for these sorts of findings, but I continue to disagree that our takeaway should be an impetus to gird ourselves for the backlash when The Machine spits out, "Black people are dumb," rather than an impetus to observe the bedrock notion of scientific inquiry (you know, skepticism). >It's easy if you're using ML to detect whether an image contains a bird or train. And it's easy when you point to an obvious example with bad data (e.g. inability to differentiate dark faces). But that's not all cases. Sometimes it's good data and the findings are unpalatable. That was the root of the question. I think the "Newspaper Error Rule" (or, I guess, the "Cockroach Rule") applies here: if you recognize one when you're looking, how many are you missing when you're not, particularly when you're in less familiar territory? The less obvious examples are the ones we should be more wary of. And if your boss is expecting a quick, cut-and-dry answer to questions that have vexed society for centuries, just because we've applied a sophisticated (but still limited) statistical model to it... I'm just saying, maybe you don't want to go down in history as the 21st century's version of a phrenologist.
- cmdshiftf4 7y ago
  >We already know what will happen when findings are unpalatable. We'll think of ways to explain why they don't matter and can't possibly be right, or that we weren't measuring the right thing in the first place. And these are just the more obvious cases! Exactly. Whether "conservative" or "progressive" in one's politics, the last decade or so has shown that when faced with a value contrary to one's own, or a bias that contradicts one's own currently held biases, one will simply dismiss the contradiction and likely hunker down on those preexisting values or biases. In the current era that is manifesting itself as who can shout the loudest and most outrageously on social media or new-media, facts and reason be damned.
thu2111 7y ago
Will we simply put in a hard override to get the answers we want to get, the answers we find comfortable Yep. That's already happening. See Google's papers on "debiasing AI" (if you look at their example goals, they mean biasing it away from what it learned and towards what Googlers wish was true).

moduspol 7y ago

> What will we do when machine learning (or science, or anything, really) comes to a conclusion we find unpalatable, for whatever reason? What makes you think it hasn't already happened? Numerous times? Biological differences between sexes? IQ differences between differentiable subsets of people? The gender equality paradox? We already know what will happen when findings are unpalatable. We'll think of ways to explain why they don't matter and can't possibly be right, or that we weren't measuring the right thing in the first place. And these are just the more obvious cases!

bsanr2 7y ago
>or that we weren't measuring the right thing in the first place. Which has been true, actually. For example, human facial recognition systems that can't properly distinguish dark-skinned faces. What is your conclusion in those cases? That dark-skinned people don't have faces? All look the same? Aren't human? Or is it more likely that the system is flawed in some way? Is that flaw in line with existing biases? What then is the chance that the flaw is BECAUSE of our existing biases, in some way or fashion? From what I've read on HN, many people involved in developing ML technology seem to be overly concerned with their systems spitting out "uncomfortable truths", and less concerned with flaws in their data capture, training, or system design processes. We're putting the cart before the horse, and the giddiness with which it's being done is troubling in and of itself.
- moduspol 7y ago
  Sure--nobody's arguing that we're always measuring the right thing. It may be more relevant in developing ML technology because readers here are the people who have to (at some point) decide whether they've found an uncomfortable truth or falsehood. Or that it's potentially biased, but may or may not be relevant. If I'm doing this at work, I may need to be able to explain to my boss whether or not what we're doing is going to be called racist or sexist, and that's regardless of whether what's being done actually is racist or sexist. It's easy if you're using ML to detect whether an image contains a bird or train. And it's easy when you point to an obvious example with bad data (e.g. inability to differentiate dark faces). But that's not all cases. Sometimes it's good data and the findings are unpalatable. That was the root of the question.
  - bsanr2 7y ago
    We say "unpalatable," but what we really mean is "politically incorrect." There are segments of our society which would be overjoyed to have such "concrete" evidence of their terrible beliefs. Therefore, it is important to observe OP's thesis: that ML "findings" are conservative; that they tend to reproduce preexisting biases; and that we should be interrogating the input data, systems, and output of ML-applied tasks that return "controversial" findings. I agree that it's important to be aware of the potential for these sorts of findings, but I continue to disagree that our takeaway should be an impetus to gird ourselves for the backlash when The Machine spits out, "Black people are dumb," rather than an impetus to observe the bedrock notion of scientific inquiry (you know, skepticism). >It's easy if you're using ML to detect whether an image contains a bird or train. And it's easy when you point to an obvious example with bad data (e.g. inability to differentiate dark faces). But that's not all cases. Sometimes it's good data and the findings are unpalatable. That was the root of the question. I think the "Newspaper Error Rule" (or, I guess, the "Cockroach Rule") applies here: if you recognize one when you're looking, how many are you missing when you're not, particularly when you're in less familiar territory? The less obvious examples are the ones we should be more wary of. And if your boss is expecting a quick, cut-and-dry answer to questions that have vexed society for centuries, just because we've applied a sophisticated (but still limited) statistical model to it... I'm just saying, maybe you don't want to go down in history as the 21st century's version of a phrenologist.
cmdshiftf4 7y ago
>We already know what will happen when findings are unpalatable. We'll think of ways to explain why they don't matter and can't possibly be right, or that we weren't measuring the right thing in the first place. And these are just the more obvious cases! Exactly. Whether "conservative" or "progressive" in one's politics, the last decade or so has shown that when faced with a value contrary to one's own, or a bias that contradicts one's own currently held biases, one will simply dismiss the contradiction and likely hunker down on those preexisting values or biases. In the current era that is manifesting itself as who can shout the loudest and most outrageously on social media or new-media, facts and reason be damned.

thu2111 7y ago

Will we simply put in a hard override to get the answers we want to get, the answers we find comfortable Yep. That's already happening. See Google's papers on "debiasing AI" (if you look at their example goals, they mean biasing it away from what it learned and towards what Googlers wish was true).