3 ms·

Question about this: they talk about how datasets are not representative, my question is, compared to what? I’m guessing by the politic of this article the per

Question about this: they talk about how datasets are not representative, my question is, compared to what? I’m guessing by the politic of this article the person is from the US, do they want people to look at data representative of US population? That seems pretty narrow-minded. My country has almost no black people (probably <1% though I don’t know that official stats on race are even collected) and it would be extremely expensive for us to include black people at US levels (13%) in all research. Perhaps they are advocating we use world demographics but that would be logistically impossible for basically all researchers around the world. Am I misunderstanding the author or does the article seem pretty ethnocentric? (is that the right word?) US-centric? Wouldn’t it be better to just qualify/label the demographics of the research data (we used all black people, all white people etc). They talk as if there is some golden ratio we should all be following but that just isn’t the case. In any case, I don’t think it is useful to shame researchers that are just doing the best they can with the data available cause the data they have is useful to someone.

BadInformatics 6y ago
For better or worse, most of these articles are US-centric because that's where most of the R&D money for health ML is. The far more reasonable approach is to make sure your data contains as many demographics as possible (not just race) from your actual patient population. If there happen to be gaps, then put in at least a reasonable effort to fill them instead of shrugging and saying "it's out of our control". That, along with a per-demographic breakdown of important metrics and your point about qualifying demographics in the data (which is already done in most medical publications, including many using ML) would already be a huge improvement on what most people do now. Ironically, it's the big tech companies that have the hardest time with this because they want to make generally deployable projects, yet don't have access to as much data as many healthcare orgs do. Frankly, I don't have much sympathy for them: a lot of this is trust issues from self-inflicted damage.
- jjcon 6y ago
  > make sure your data contains as many demographics as possible (not just race) from your actual patient population By this you mean researchers should mirror their patient population as closely as possible (be it socioeconomic, gender, race etc) in whatever region they may operate in (which may not scale perfectly worldwide but will serve patients in that region well)?

BadInformatics 6y ago

For better or worse, most of these articles are US-centric because that's where most of the R&D money for health ML is. The far more reasonable approach is to make sure your data contains as many demographics as possible (not just race) from your actual patient population. If there happen to be gaps, then put in at least a reasonable effort to fill them instead of shrugging and saying "it's out of our control". That, along with a per-demographic breakdown of important metrics and your point about qualifying demographics in the data (which is already done in most medical publications, including many using ML) would already be a huge improvement on what most people do now. Ironically, it's the big tech companies that have the hardest time with this because they want to make generally deployable projects, yet don't have access to as much data as many healthcare orgs do. Frankly, I don't have much sympathy for them: a lot of this is trust issues from self-inflicted damage.

jjcon 6y ago
> make sure your data contains as many demographics as possible (not just race) from your actual patient population By this you mean researchers should mirror their patient population as closely as possible (be it socioeconomic, gender, race etc) in whatever region they may operate in (which may not scale perfectly worldwide but will serve patients in that region well)?