4 ms·

Sure--nobody's arguing that we're always measuring the right thing. It may be more relevant in developing ML technology because readers here are the people who

Sure--nobody's arguing that we're always measuring the right thing. It may be more relevant in developing ML technology because readers here are the people who have to (at some point) decide whether they've found an uncomfortable truth or falsehood. Or that it's potentially biased, but may or may not be relevant. If I'm doing this at work, I may need to be able to explain to my boss whether or not what we're doing is going to be called racist or sexist, and that's regardless of whether what's being done actually is racist or sexist. It's easy if you're using ML to detect whether an image contains a bird or train. And it's easy when you point to an obvious example with bad data (e.g. inability to differentiate dark faces). But that's not all cases. Sometimes it's good data and the findings are unpalatable. That was the root of the question.

bsanr2 7y ago
We say "unpalatable," but what we really mean is "politically incorrect." There are segments of our society which would be overjoyed to have such "concrete" evidence of their terrible beliefs. Therefore, it is important to observe OP's thesis: that ML "findings" are conservative; that they tend to reproduce preexisting biases; and that we should be interrogating the input data, systems, and output of ML-applied tasks that return "controversial" findings. I agree that it's important to be aware of the potential for these sorts of findings, but I continue to disagree that our takeaway should be an impetus to gird ourselves for the backlash when The Machine spits out, "Black people are dumb," rather than an impetus to observe the bedrock notion of scientific inquiry (you know, skepticism). >It's easy if you're using ML to detect whether an image contains a bird or train. And it's easy when you point to an obvious example with bad data (e.g. inability to differentiate dark faces). But that's not all cases. Sometimes it's good data and the findings are unpalatable. That was the root of the question. I think the "Newspaper Error Rule" (or, I guess, the "Cockroach Rule") applies here: if you recognize one when you're looking, how many are you missing when you're not, particularly when you're in less familiar territory? The less obvious examples are the ones we should be more wary of. And if your boss is expecting a quick, cut-and-dry answer to questions that have vexed society for centuries, just because we've applied a sophisticated (but still limited) statistical model to it... I'm just saying, maybe you don't want to go down in history as the 21st century's version of a phrenologist.
- moduspol 7y ago
  I think we mostly agree on premise but are disagreeing on emphasis. > I agree that it's important to be aware of the potential for these sorts of findings, but I continue to disagree that our takeaway should be an impetus to gird ourselves for the backlash when The Machine spits out, "Black people are dumb," rather than an impetus to observe the bedrock notion of scientific inquiry (you know, skepticism). There is no shortage of pressure to avoid ML conclusions like "black people are dumb." We are already girded there. This isn't the first HN article about recognizing ML biases, and it's not like we're wading through headlines about how racial stereotypes are justified due to some all-knowing ML algorithm. We know we should be interrogating biases. That said, it will always be possible to dismiss any evidence-based science by claiming there are unfalsifiable flaws with data or methods. That was the point of the comment I responded to. Outside the context of ML, this already happens, so we can already predict how it will happen with ML. And it is. Nobody's saying, "just trust the machine if it says black people are dumb." I'm saying, "if the machine says sickle-cell anemia is most common among those with sub-Saharan African ancestry, it's still worth checking your methods and data, but it shouldn't be ignored just because unfalsifiable reasons why the analysis could be wrong can be postulated."
  - bsanr2 7y ago
    The tendency to say, "Don't worry, we've got it covered," is precisely where my worry stems from. It is better than not having any awareness of the potential problems, to be sure, but only so much so. Your last analogy is an example of this. The link between sickle cell anemia and ancestry is a causal link that could be shown conclusively with a genetic test for a SNP. Finding that gene may have been an involved effort undertaken with a great many resources, but that link was established with near-perfect data and a clear conclusion. The most promising are applications for ML are situations on the exact opposite end of the spectrum: problems where data is imperfect, noisy, or incomplete, where conclusions are not certain, but simply likely, to some measure of confidence. We want to use ML to come to answers approaching conclusive, when a solid conclusion is impossible or unlikely, or would too long to obtain by conventional means. And we won't know how that conclusion is reached. So, in terms of conclusions that could spark wars, genocide, untold suffering, maybe we need to be more serious about our vigilance in regards to what we can control. To put it bluntly, these concerns people have seem akin to NASA being more worried about handling post-catastrophe PR and internal messaging than keeping the shuttle from exploding in the first place.

bsanr2 7y ago

We say "unpalatable," but what we really mean is "politically incorrect." There are segments of our society which would be overjoyed to have such "concrete" evidence of their terrible beliefs. Therefore, it is important to observe OP's thesis: that ML "findings" are conservative; that they tend to reproduce preexisting biases; and that we should be interrogating the input data, systems, and output of ML-applied tasks that return "controversial" findings. I agree that it's important to be aware of the potential for these sorts of findings, but I continue to disagree that our takeaway should be an impetus to gird ourselves for the backlash when The Machine spits out, "Black people are dumb," rather than an impetus to observe the bedrock notion of scientific inquiry (you know, skepticism). >It's easy if you're using ML to detect whether an image contains a bird or train. And it's easy when you point to an obvious example with bad data (e.g. inability to differentiate dark faces). But that's not all cases. Sometimes it's good data and the findings are unpalatable. That was the root of the question. I think the "Newspaper Error Rule" (or, I guess, the "Cockroach Rule") applies here: if you recognize one when you're looking, how many are you missing when you're not, particularly when you're in less familiar territory? The less obvious examples are the ones we should be more wary of. And if your boss is expecting a quick, cut-and-dry answer to questions that have vexed society for centuries, just because we've applied a sophisticated (but still limited) statistical model to it... I'm just saying, maybe you don't want to go down in history as the 21st century's version of a phrenologist.

moduspol 7y ago
I think we mostly agree on premise but are disagreeing on emphasis. > I agree that it's important to be aware of the potential for these sorts of findings, but I continue to disagree that our takeaway should be an impetus to gird ourselves for the backlash when The Machine spits out, "Black people are dumb," rather than an impetus to observe the bedrock notion of scientific inquiry (you know, skepticism). There is no shortage of pressure to avoid ML conclusions like "black people are dumb." We are already girded there. This isn't the first HN article about recognizing ML biases, and it's not like we're wading through headlines about how racial stereotypes are justified due to some all-knowing ML algorithm. We know we should be interrogating biases. That said, it will always be possible to dismiss any evidence-based science by claiming there are unfalsifiable flaws with data or methods. That was the point of the comment I responded to. Outside the context of ML, this already happens, so we can already predict how it will happen with ML. And it is. Nobody's saying, "just trust the machine if it says black people are dumb." I'm saying, "if the machine says sickle-cell anemia is most common among those with sub-Saharan African ancestry, it's still worth checking your methods and data, but it shouldn't be ignored just because unfalsifiable reasons why the analysis could be wrong can be postulated."
- bsanr2 7y ago
  The tendency to say, "Don't worry, we've got it covered," is precisely where my worry stems from. It is better than not having any awareness of the potential problems, to be sure, but only so much so. Your last analogy is an example of this. The link between sickle cell anemia and ancestry is a causal link that could be shown conclusively with a genetic test for a SNP. Finding that gene may have been an involved effort undertaken with a great many resources, but that link was established with near-perfect data and a clear conclusion. The most promising are applications for ML are situations on the exact opposite end of the spectrum: problems where data is imperfect, noisy, or incomplete, where conclusions are not certain, but simply likely, to some measure of confidence. We want to use ML to come to answers approaching conclusive, when a solid conclusion is impossible or unlikely, or would too long to obtain by conventional means. And we won't know how that conclusion is reached. So, in terms of conclusions that could spark wars, genocide, untold suffering, maybe we need to be more serious about our vigilance in regards to what we can control. To put it bluntly, these concerns people have seem akin to NASA being more worried about handling post-catastrophe PR and internal messaging than keeping the shuttle from exploding in the first place.