3 ms·

One Harvard study trained an AI that could reliably determine someone's race from a chest X-ray. AIs can be trained to see things we can't. The difficulty is l

One Harvard study trained an AI that could reliably determine someone's race from a chest X-ray. AIs can be trained to see things we can't. The difficulty is likely in making a good training dataset of labeled images with pathologies radiologists couldn't see. I imagine in some cases (like cancer), you may happen to have an earlier CT scan or X-ray from the patient where the pathology is not quite yet detectable to the human eye.

devilbunny 1y ago
I suspect that radiologists could identify race from a plain chest x-ray if they were given the patient’s race and asked to start noticing the difference. They just aren’t doing it because, if it’s important, you can just look at the patient. There are a lot of things in medicine that aren’t in literature, but are well-known among certain practitioners. I’m an anesthesiologist and practice in an area with a large African-American population. About 10-15% (rough guess) of people of West African descent will have a ridiculously strong salivary response to certain drugs (muscarinic agonists). As in, after one dose their mouths will be full of saliva in seconds. We don’t have East Africans for comparison, so I can’t say it’s a pan-Bantu thing, but I have seen it in a Nigerian who lived here. Not in the literature, but we all know it. I had a (EDIT: non-anesthesia) colleague ask me about a hypersecretory response from such a drug. I said, oh, was he black? Yes, how did I know? Because we give those drugs all the time and have eyes. It’s very rare to see in European-descended populations.
- nearbuy 1y ago
  It's possible humans could learn to do this, but I'm skeptical they could do it this well. According to the article, humans experts couldn't tell race from the chest X-rays and the researchers couldn't figure out how the AI was detecting race. They fed it corrupted images to figure out what information it was relying on. It was robust against both low pass and high pass filters.
bilbo0s 1y ago
Sigh. Rads can see a person's race. We look at them. That's the reason rads never train to determine race from a chest x-ray. BTW, models don't need to train that either. Because if it's important, it's recorded, along with a picture, in the guy's medical record. I'd just like to gently suggest that determining someone's race from an X-Ray instead of, say, their photograph, is maybe not how we should be burning training cycles if we want to push medical imaging forward. Human radiologists had that figured out ages ago.
- deleted 1y ago
  [deleted]
- nearbuy 1y ago
  You're being snide about the Harvard/MIT researchers being idiots doing useless research because they don't realize radiologists can just look at the patients face, but that's obviously not what happened. They were trying to see if AI could introduce racial bias. They're not releasing an AI to tell radiologists the race of their patients. According to the article, human experts could not tell race from chest X-rays, while the AI could to do reliably. Further, it could still determine race when given an X-ray image passed through a low pass filter or high pass filter, showing that it's not relying solely on fine detail or large features to do so.
  - bilbo0s 1y ago
    Sigh. Firstly, that doesn't tell us whether there is bias in data. That tells us whether or not there is bias in their data. Secondly, it tells us it can train to spot things that human rads do not train to spot. It tells us nothing at all about whether or not an AI can train to spot things a human rad also trains to spot, but can't. Human rads don't train to spot race. Why? Because they don't need to do so. Human rads do train to spot pathologies in as early a stage as possible. I've never seen an AI spot one at an earlier stage than the best human rads can. But I have seen several AIs fail to spot pathologies that even human rads at the median could spot. That's the state of play today. And it's likely to remain that way for a long, long time. Human rads will be needed to review this work not because they are human, but right now, it's because human rads are just better. At the top end, human rads are not only better, but are manifestly superior.

devilbunny 1y ago

I suspect that radiologists could identify race from a plain chest x-ray if they were given the patient’s race and asked to start noticing the difference. They just aren’t doing it because, if it’s important, you can just look at the patient. There are a lot of things in medicine that aren’t in literature, but are well-known among certain practitioners. I’m an anesthesiologist and practice in an area with a large African-American population. About 10-15% (rough guess) of people of West African descent will have a ridiculously strong salivary response to certain drugs (muscarinic agonists). As in, after one dose their mouths will be full of saliva in seconds. We don’t have East Africans for comparison, so I can’t say it’s a pan-Bantu thing, but I have seen it in a Nigerian who lived here. Not in the literature, but we all know it. I had a (EDIT: non-anesthesia) colleague ask me about a hypersecretory response from such a drug. I said, oh, was he black? Yes, how did I know? Because we give those drugs all the time and have eyes. It’s very rare to see in European-descended populations.

nearbuy 1y ago
It's possible humans could learn to do this, but I'm skeptical they could do it this well. According to the article, humans experts couldn't tell race from the chest X-rays and the researchers couldn't figure out how the AI was detecting race. They fed it corrupted images to figure out what information it was relying on. It was robust against both low pass and high pass filters.

bilbo0s 1y ago

Sigh. Rads can see a person's race. We look at them. That's the reason rads never train to determine race from a chest x-ray. BTW, models don't need to train that either. Because if it's important, it's recorded, along with a picture, in the guy's medical record. I'd just like to gently suggest that determining someone's race from an X-Ray instead of, say, their photograph, is maybe not how we should be burning training cycles if we want to push medical imaging forward. Human radiologists had that figured out ages ago.

deleted 1y ago
[deleted]
nearbuy 1y ago
You're being snide about the Harvard/MIT researchers being idiots doing useless research because they don't realize radiologists can just look at the patients face, but that's obviously not what happened. They were trying to see if AI could introduce racial bias. They're not releasing an AI to tell radiologists the race of their patients. According to the article, human experts could not tell race from chest X-rays, while the AI could to do reliably. Further, it could still determine race when given an X-ray image passed through a low pass filter or high pass filter, showing that it's not relying solely on fine detail or large features to do so.
- bilbo0s 1y ago
  Sigh. Firstly, that doesn't tell us whether there is bias in data. That tells us whether or not there is bias in their data. Secondly, it tells us it can train to spot things that human rads do not train to spot. It tells us nothing at all about whether or not an AI can train to spot things a human rad also trains to spot, but can't. Human rads don't train to spot race. Why? Because they don't need to do so. Human rads do train to spot pathologies in as early a stage as possible. I've never seen an AI spot one at an earlier stage than the best human rads can. But I have seen several AIs fail to spot pathologies that even human rads at the median could spot. That's the state of play today. And it's likely to remain that way for a long, long time. Human rads will be needed to review this work not because they are human, but right now, it's because human rads are just better. At the top end, human rads are not only better, but are manifestly superior.