3 ms·

> I mean the dataset is so small, one off can derail any conclusion What do you mean? Per the Supplement, their data comes from a pool of ~4k white doctors and

> I mean the dataset is so small, one off can derail any conclusion What do you mean? Per the Supplement, their data comes from a pool of ~4k white doctors and ~500 black doctors, with hundreds of thousands of observations/patients. This dataset doesn't seem "so small"? [1] https://www.pnas.org/content/pnas/suppl/2020/08/12/1913405117.DCSupplemental/pnas.1913405117.sapp.pdf https://www.pnas.org/content/pnas/suppl/2020/08/12/191340511...

sz4kerto 6y ago
It really all depends on the methodology. I recommend some reading about 'p-hacking'. It is not that hard to find to find really shocking findings in large datasets -- the question is how hard you look. In this particular case, the question is how many hypotheses have been evaluated, i.e. has the author 'mined the data' for correlations. If yes, then the acceptable p value is much smaller than normal.
- croissants 6y ago
  I think it's a bit of a reach to suggest that this is p-hacking. Some of the relevant authors have studied this hypothesis (a gap in pregnancy-related outcomes between black and white people) across multiple papers, and the effect found here -- ignoring any p calculation -- is pretty big. It's not like they're picking some extremely specific combination of factors.
  - x86_64Ubuntu 6y ago
    And this is the narrative that I'm speaking of when I mention how these discussions go. It's a gish-gallop of saying that since we have imperfect knowledge, we can't draw a single conclusion about phenomena and systems that are well documented. First it's the lack of studies, then you get a study and it's a hidden bias in the populations used in the study, then it's the "low sample size" (that's a favorite), after that it's p-hacking or whatever else. And it goes on and on, even though for hundreds of years, the country and society were tuned to ensure that the black community enjoyed degraded outcomes in all aspects of life.
deleted 6y ago
[deleted]
AzzieElbab 6y ago
Thanks for posting this, I must have misread the original article. The samples are x10 larger than I understood at first. However, please keep in mind that the subset of white ph/patients is almost 4 times larger than black ph/black patient. Data fitting seem questionable from there on. 4x is not that large of a scale but still analyses should have been more nuanced from here. For example, hospitals are not considered a variable at all. I would think that it is a major one
- jjeaff 6y ago
  Are you sure you don't just have some unconscious bias that is keeping you from objectively seeing this study for what it is? There are about 4x more white people in America than Black. So it makes perfect sense that you would have 4x as many white patients as Black in the study.
  - AzzieElbab 6y ago
    I am saying that in order to be accurate the report needs to be tied to specific doctors or at the very latest least specific hospitals.

sz4kerto 6y ago

It really all depends on the methodology. I recommend some reading about 'p-hacking'. It is not that hard to find to find really shocking findings in large datasets -- the question is how hard you look. In this particular case, the question is how many hypotheses have been evaluated, i.e. has the author 'mined the data' for correlations. If yes, then the acceptable p value is much smaller than normal.

croissants 6y ago
I think it's a bit of a reach to suggest that this is p-hacking. Some of the relevant authors have studied this hypothesis (a gap in pregnancy-related outcomes between black and white people) across multiple papers, and the effect found here -- ignoring any p calculation -- is pretty big. It's not like they're picking some extremely specific combination of factors.
- x86_64Ubuntu 6y ago
  And this is the narrative that I'm speaking of when I mention how these discussions go. It's a gish-gallop of saying that since we have imperfect knowledge, we can't draw a single conclusion about phenomena and systems that are well documented. First it's the lack of studies, then you get a study and it's a hidden bias in the populations used in the study, then it's the "low sample size" (that's a favorite), after that it's p-hacking or whatever else. And it goes on and on, even though for hundreds of years, the country and society were tuned to ensure that the black community enjoyed degraded outcomes in all aspects of life.

deleted 6y ago

[deleted]

AzzieElbab 6y ago

Thanks for posting this, I must have misread the original article. The samples are x10 larger than I understood at first. However, please keep in mind that the subset of white ph/patients is almost 4 times larger than black ph/black patient. Data fitting seem questionable from there on. 4x is not that large of a scale but still analyses should have been more nuanced from here. For example, hospitals are not considered a variable at all. I would think that it is a major one

jjeaff 6y ago
Are you sure you don't just have some unconscious bias that is keeping you from objectively seeing this study for what it is? There are about 4x more white people in America than Black. So it makes perfect sense that you would have 4x as many white patients as Black in the study.
- AzzieElbab 6y ago
  I am saying that in order to be accurate the report needs to be tied to specific doctors or at the very latest least specific hospitals.