4 ms·

I think we mostly agree on premise but are disagreeing on emphasis. > I agree that it's important to be aware of the potential for these sorts of findings, but

I think we mostly agree on premise but are disagreeing on emphasis. > I agree that it's important to be aware of the potential for these sorts of findings, but I continue to disagree that our takeaway should be an impetus to gird ourselves for the backlash when The Machine spits out, "Black people are dumb," rather than an impetus to observe the bedrock notion of scientific inquiry (you know, skepticism). There is no shortage of pressure to avoid ML conclusions like "black people are dumb." We are already girded there. This isn't the first HN article about recognizing ML biases, and it's not like we're wading through headlines about how racial stereotypes are justified due to some all-knowing ML algorithm. We know we should be interrogating biases. That said, it will always be possible to dismiss any evidence-based science by claiming there are unfalsifiable flaws with data or methods. That was the point of the comment I responded to. Outside the context of ML, this already happens, so we can already predict how it will happen with ML. And it is. Nobody's saying, "just trust the machine if it says black people are dumb." I'm saying, "if the machine says sickle-cell anemia is most common among those with sub-Saharan African ancestry, it's still worth checking your methods and data, but it shouldn't be ignored just because unfalsifiable reasons why the analysis could be wrong can be postulated."

bsanr2 7y ago
The tendency to say, "Don't worry, we've got it covered," is precisely where my worry stems from. It is better than not having any awareness of the potential problems, to be sure, but only so much so. Your last analogy is an example of this. The link between sickle cell anemia and ancestry is a causal link that could be shown conclusively with a genetic test for a SNP. Finding that gene may have been an involved effort undertaken with a great many resources, but that link was established with near-perfect data and a clear conclusion. The most promising are applications for ML are situations on the exact opposite end of the spectrum: problems where data is imperfect, noisy, or incomplete, where conclusions are not certain, but simply likely, to some measure of confidence. We want to use ML to come to answers approaching conclusive, when a solid conclusion is impossible or unlikely, or would too long to obtain by conventional means. And we won't know how that conclusion is reached. So, in terms of conclusions that could spark wars, genocide, untold suffering, maybe we need to be more serious about our vigilance in regards to what we can control. To put it bluntly, these concerns people have seem akin to NASA being more worried about handling post-catastrophe PR and internal messaging than keeping the shuttle from exploding in the first place.
- moduspol 7y ago
  Your perspective doesn't lead to the discovery of the true causal link for sickle cell anemia, though. It's what leads to us pretending there can't be any unpalatable differences between ethnic groups, clutching to presumptions about maybe the health care system being too racist, and avoiding looking deeper into it because we think we've already decided what the answer is (racism). Meanwhile real people are suffering because they aren't getting treated as effectively as they could. We don't know what we don't know, so it's impossible to tell how many people are suffering or for what reasons.
  - bsanr2 7y ago
    We know what racism begets. We've seen the way that it has been used to warp inquiry, ethical guidelines, and clinical outcomes, often unintentionally and through processes that diffuse blame to below the level of individual bigotry. It would be nice if we lived in a world without a track record of pursuing the ramifications of assumed "unpalatable differences," such that what you're suggesting would be sound, but we don't. We, at this moment, are trying to climb out of a hole of ignorance - in medicine, in social science, in economics, in much of the quantifiable and model-able world - dug by presumptions of "unpalatable differences." So, I say again, the the priorities, as you've described them, are out of wack. We are much more at risk of jumping to damaging conclusions than we are of missing helpful breakthroughs. Our vigilance should be tuned in regard to this.

bsanr2 7y ago

The tendency to say, "Don't worry, we've got it covered," is precisely where my worry stems from. It is better than not having any awareness of the potential problems, to be sure, but only so much so. Your last analogy is an example of this. The link between sickle cell anemia and ancestry is a causal link that could be shown conclusively with a genetic test for a SNP. Finding that gene may have been an involved effort undertaken with a great many resources, but that link was established with near-perfect data and a clear conclusion. The most promising are applications for ML are situations on the exact opposite end of the spectrum: problems where data is imperfect, noisy, or incomplete, where conclusions are not certain, but simply likely, to some measure of confidence. We want to use ML to come to answers approaching conclusive, when a solid conclusion is impossible or unlikely, or would too long to obtain by conventional means. And we won't know how that conclusion is reached. So, in terms of conclusions that could spark wars, genocide, untold suffering, maybe we need to be more serious about our vigilance in regards to what we can control. To put it bluntly, these concerns people have seem akin to NASA being more worried about handling post-catastrophe PR and internal messaging than keeping the shuttle from exploding in the first place.

moduspol 7y ago
Your perspective doesn't lead to the discovery of the true causal link for sickle cell anemia, though. It's what leads to us pretending there can't be any unpalatable differences between ethnic groups, clutching to presumptions about maybe the health care system being too racist, and avoiding looking deeper into it because we think we've already decided what the answer is (racism). Meanwhile real people are suffering because they aren't getting treated as effectively as they could. We don't know what we don't know, so it's impossible to tell how many people are suffering or for what reasons.
- bsanr2 7y ago
  We know what racism begets. We've seen the way that it has been used to warp inquiry, ethical guidelines, and clinical outcomes, often unintentionally and through processes that diffuse blame to below the level of individual bigotry. It would be nice if we lived in a world without a track record of pursuing the ramifications of assumed "unpalatable differences," such that what you're suggesting would be sound, but we don't. We, at this moment, are trying to climb out of a hole of ignorance - in medicine, in social science, in economics, in much of the quantifiable and model-able world - dug by presumptions of "unpalatable differences." So, I say again, the the priorities, as you've described them, are out of wack. We are much more at risk of jumping to damaging conclusions than we are of missing helpful breakthroughs. Our vigilance should be tuned in regard to this.