4 ms·

> What will we do when machine learning (or science, or anything, really) comes to a conclusion we find unpalatable, for whatever reason? What makes you think

• bsanr2 7y ago
  >or that we weren't measuring the right thing in the first place. Which has been true, actually. For example, human facial recognition systems that can't properly distinguish dark-skinned faces. What is your conclusion in those cases? That dark-skinned people don't have faces? All look the same? Aren't human? Or is it more likely that the system is flawed in some way? Is that flaw in line with existing biases? What then is the chance that the flaw is BECAUSE of our existing biases, in some way or fashion? From what I've read on HN, many people involved in developing ML technology seem to be overly concerned with their systems spitting out "uncomfortable truths", and less concerned with flaws in their data capture, training, or system design processes. We're putting the cart before the horse, and the giddiness with which it's being done is troubling in and of itself.
  • moduspol 7y ago
    Sure--nobody's arguing that we're always measuring the right thing. It may be more relevant in developing ML technology because readers here are the people who have to (at some point) decide whether they've found an uncomfortable truth or falsehood. Or that it's potentially biased, but may or may not be relevant. If I'm doing this at work, I may need to be able to explain to my boss whether or not what we're doing is going to be called racist or sexist, and that's regardless of whether what's being done actually is racist or sexist. It's easy if you're using ML to detect whether an image contains a bird or train. And it's easy when you point to an obvious example with bad data (e.g. inability to differentiate dark faces). But that's not all cases. Sometimes it's good data and the findings are unpalatable. That was the root of the question.
    • bsanr2 7y ago
      We say "unpalatable," but what we really mean is "politically incorrect." There are segments of our society which would be overjoyed to have such "concrete" evidence of their terrible beliefs. Therefore, it is important to observe OP's thesis: that ML "findings" are conservative; that they tend to reproduce preexisting biases; and that we should be interrogating the input data, systems, and output of ML-applied tasks that return "controversial" findings. I agree that it's important to be aware of the potential for these sorts of findings, but I continue to disagree that our takeaway should be an impetus to gird ourselves for the backlash when The Machine spits out, "Black people are dumb," rather than an impetus to observe the bedrock notion of scientific inquiry (you know, skepticism). >It's easy if you're using ML to detect whether an image contains a bird or train. And it's easy when you point to an obvious example with bad data (e.g. inability to differentiate dark faces). But that's not all cases. Sometimes it's good data and the findings are unpalatable. That was the root of the question. I think the "Newspaper Error Rule" (or, I guess, the "Cockroach Rule") applies here: if you recognize one when you're looking, how many are you missing when you're not, particularly when you're in less familiar territory? The less obvious examples are the ones we should be more wary of. And if your boss is expecting a quick, cut-and-dry answer to questions that have vexed society for centuries, just because we've applied a sophisticated (but still limited) statistical model to it... I'm just saying, maybe you don't want to go down in history as the 21st century's version of a phrenologist.
      • moduspol 7y ago
        I think we mostly agree on premise but are disagreeing on emphasis. > I agree that it's important to be aware of the potential for these sorts of findings, but I continue to disagree that our takeaway should be an impetus to gird ourselves for the backlash when The Machine spits out, "Black people are dumb," rather than an impetus to observe the bedrock notion of scientific inquiry (you know, skepticism). There is no shortage of pressure to avoid ML conclusions like "black people are dumb." We are already girded there. This isn't the first HN article about recognizing ML biases, and it's not like we're wading through headlines about how racial stereotypes are justified due to some all-knowing ML algorithm. We know we should be interrogating biases. That said, it will always be possible to dismiss any evidence-based science by claiming there are unfalsifiable flaws with data or methods. That was the point of the comment I responded to. Outside the context of ML, this already happens, so we can already predict how it will happen with ML. And it is. Nobody's saying, "just trust the machine if it says black people are dumb." I'm saying, "if the machine says sickle-cell anemia is most common among those with sub-Saharan African ancestry, it's still worth checking your methods and data, but it shouldn't be ignored just because unfalsifiable reasons why the analysis could be wrong can be postulated."
• cmdshiftf4 7y ago
  >We already know what will happen when findings are unpalatable. We'll think of ways to explain why they don't matter and can't possibly be right, or that we weren't measuring the right thing in the first place. And these are just the more obvious cases! Exactly. Whether "conservative" or "progressive" in one's politics, the last decade or so has shown that when faced with a value contrary to one's own, or a bias that contradicts one's own currently held biases, one will simply dismiss the contradiction and likely hunker down on those preexisting values or biases. In the current era that is manifesting itself as who can shout the loudest and most outrageously on social media or new-media, facts and reason be damned.