10 ms·
The dataset they used to train the model are chest xrays of known diseases. I'm having trouble understanding how that's relevant here. The key takeaway is that
by MountainArras 2y ago
The dataset they used to train the model are chest xrays of known diseases. I'm having trouble understanding how that's relevant here. The key takeaway is that you can't treat all humans as a single group in this context, and variations in the biology across different groups of people may need to be taken into account within the training process. In other words, the model will need to be trained on this racial/gender data too in order to get better results when predicting the targeted diseases within these groups.
I think it's interesting to think about instead attaching generic information instead of group data, which would be blind to human bias and the messiness of our rough categorizations of subgroups.
- pelorat 2y agoI think the model needs to be thought about human anatomy, not just fed a bunch of scans. It needs to understand what ribs and organs are.
- ericmcer 2y agoI don't think LLMs can achieve "understanding" in that sense.
- nomel 2y agoThese aren't LLM. Most of the neat things in science, involving AI, aren't LLM. Next word prediction has extremely limited use with non-text data.
- thaumasiotes 2y agoPeople seem to have started to use "LLM" to refer to any suite of software that includes an LLM somewhere within it; you can see them talking about LLM-generated art, for example.
- hnlmorg 2y agoWas it ascii art? ;)
- thaumasiotes 2y agohttps://hamatti.org/posts/art-forgery-llms-and-why-it-feels-a-bit-off/ https://hamatti.org/posts/art-forgery-llms-and-why-it-feels-... People will just believe whatever they hear.
- satvikpendem 2y agoComputer vision models are not large language models; LLM does not mean generative AI or even AI in general, it stands for a specific initialism.
- bko 2y agoApparently providing this messy rough categorization appeared to help in some cases. From the article: > To force CheXzero to avoid shortcuts and therefore try to mitigate this bias, the team repeated the experiment but deliberately gave the race, sex, or age of patients to the model together with the images. The model’s rate of “missed” diagnoses decreased by half—but only for some conditions. In the end though I think you're right and we're just at the phases of hand-coding attributes. The bitter lesson always prevails https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdf https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson...
- thaumasiotes 2y ago> Also important was the use [in Go] of learning by self play to learn a value function I thought the self-play was the value function that made progress in Go. That is, it wasn't the case that we played through a lot of games and used that data to create a function that would assign a value to a Go board. Instead, the function to assign a value to a Go board would do some self-play on the board and assign value based on the outcome.
- ruytlm 2y agoIt disappoints me how easily we are collectively falling for what effectively is "Oh, our model is biased, but the only way to fix it is that everyone needs to give us all their data, so that we can eliminate that bias. If you think the model shouldn't be biased, you're morally obligated to give us everything you have for free. Oh but then we'll charge you for the outputs." How convenient. It's increasingly looking like the AI business model is "rent extracting middleman", just like the Elseviers et al of the academic publishing world - wedging themselves into a position where they get to take everything for free, but charge others at every opportunity.
- guhwhut 2y ago[dead]
- ElevenLathe 2y agoWe have to invent more ways to pay rich people for being rich, and AI looks like a promising one.
- genocidicbunny 2y agoDo you think there is a middle ground for a progressive 'detailization' of the data -- you form a model based on the minimal data set that allows you to draw useful conclusions, and refine that with additional data to where you're capturing the vast majority of the problem space with minimal bias?
- multjoy 2y agoThe key takeaway from the article is that the race etc. of the subjects wasn't disclosed to the AI, yet it was able to predict it to 80% while the human experts managed 50% suggesting that there was something else encoded in the imagery that the AI was picking up on.
- mjevans 2y agoThe AI might just have a better subjective / analytical weight detection criteria. Humans are likely more willing to see what they (or not see what they don't) expect to see.
- genocidicbunny 2y agoOne of the things that people I know in the medical field have mentioned is that there's racial and gender bias that goes through all levels and has a sort of feedback loop. A lot of medical knowledge is gained empirically, and historically that has meant that minorities and women tended to be underrepresented in western medical literature. That leads to new medical practitioners being less exposed to presentations of various ailments that may have variance due to gender or ethnicity. Basically, if most data is gathered from those who have the most access to medicine, there will be an inherent bias towards how various ailments present in those populations. So your base data set might be skewed from the very beginning. (This is mostly just to offer some food for thought, I haven't read the article in full so I don't want to comment on it specifically.)
- belorn 2y agoIt is very true that a lot of medical knowledge is gained empirically, and there is also an additional aspect to it. The history of Medical research is generally studied on the demographics where such testing is cultural acceptable, and where the gains of such research has been mostly sought, which is young men drafted into wars. The second common demographic are medical students, which historically was biased towards men but are today biased towards women. So while access to medicine indeed one demographic, I would say that studies are more likely to target demographics which are convenient to test on.
- genocidicbunny 2y agoI think we're really talking about different aspects of the same issue. Everything you've described basically agrees with "those who have more access to medicine" because those are also the ones inherently more convenient to test/observe.
- klipt 2y agoLike how the ones with the most access to medicine are mice, because they're convenient to experiment on.
- dartos 2y ago> The dataset they used to train the model are chest xrays of known diseases. I'm having trouble understanding how that's relevant here. For example, If you include no (or few enough) black women in the dataset of x-rays, the model may very well miss signs of disease in black women. The biases and mistakes of those who created the data set leak into the model. Early image recognition models had some very… culturally insensitive classes baked in.
- prasadjoglekar 2y agoXays by definition don't look at skin color. Do chest x-rays of black women reveal that there's something different about their chests than white or asian women? That doesn't pass my non doctor sniff test, but someone can correct me (no sarcasm intended).
- CJefferson 2y agoThis is the whole point of the article. Did you read it? Does the whole thing fail your sniff test? Their results seem solid, and clear, to me.
- guhwhut 2y agoCancer progresses differently depending on ethnicity and sex. As does treatment and likelihood of receiving treatment at early stages. Black women experience worse outcomes and are diagnosed with more severe forms of breast cancer than white women. Cancer is not just one disease. Its progression will vary depending on type. If the AI is trained on only some strains of cancer, eg those traditionally found in white women in early detection scenarios, it might not generalize to other cancer types. So yes, to your genuine question, medical imaging of cancer can vary depending on ethnicity because different cancers can vary between genetic backgrounds. Ideally there would be sufficient training data across the populations, but there isn't because of historical race bias. (Among other reasons.)
- sc68cal 2y agoWhat groups have the financial means to get chest x-rays, and what groups do not? What historical events could create the circumstances where different groups have different health outcomes?
- darkerside 2y agoDo you mean genetic information?
- deleted 2y ago[deleted]
- loa_in_ 2y agoX-rays are ordered only after doctor decides it's recommended. If there's dismissal bias in the decision tree at that point, many ill chests are missing from training data.