3 ms·

> This means that 22 out of 24 studies found positive associations. > The authors present a model that says there should be more papers showing negative effect

by fouc 5y ago

> This means that 22 out of 24 studies found positive associations. > The authors present a model that says there should be more papers showing negative effects just by chance. To the layperson it sure looks like the lead-crime hypothesis is confirmed. Really curious to see a more in-depth follow-up on this meta-analysis.

HWR_14 5y ago
It's interesting because, as I understand it (based solely on the article's summary), the meta-analysis asserts that the current lead-crime value is wrong. Either it is much lower than expected because we should see more ambiguous papers and the lack of them points to bias in the study/publish methodology -or- its much stronger than expected because we cannot find many counterexamples. The original publishers just went with the first, the article asserts the second.
- corty 5y ago
  I think both conclusions are wrong. Since this is a meta-study, drawing conclusions from the absence of null-results and the absence of clustering is prone to a lot of confounders. The article mentions publication-bias, where non-null-results are published with a higher probability. Other such confounders will exist, e.g. data-availability-bias in certain places and populations, location-bias because researchers study their usually first-world, urban, university-town surroundings more often. Community-bias, because researchers tend to cluster and get inspiration for new studies from their community and peer group on the conferences they visit. Data-similarity-bias, because they can only compare studies where data can be normalized to a common base. Quality-bias, because they necessarily include well-done and badly-done studies, in different amounts, usually without weighing them properly, because that is very hard to do. Most of those confounders cannot be corrected for. So one could draw a conclusion like "we do see null-results in the expected amount and clustering in the expected amount, therefore the result of this meta-study is a confirmation of the lead-crime-hypothesis". But the only correct opposite conclusion is "we do not see null-results and clustering in the expected amount, so our result is 'we do not know'". So the imho correct interpretation of the meta-study is "don't know, further research or better meta-analysis needed".
  - ectopod 5y ago
    Don't forget outright fraud. https://news.ycombinator.com/item?id=27884233 https://news.ycombinator.com/item?id=27884233
paulpauper 5y ago
not really. the lead-crime hypothesis posits that the reduction of crime in the 90s is due to the reduction of lead, but if the effect is really tiny, which the paper suggests, then it likely means that other factors, such as increased incarceration and or accessibility to abortion, may also be to blame for falling crime, or something else entirely.

by fouc 5y ago

HWR_14 5y ago

It's interesting because, as I understand it (based solely on the article's summary), the meta-analysis asserts that the current lead-crime value is wrong. Either it is much lower than expected because we should see more ambiguous papers and the lack of them points to bias in the study/publish methodology -or- its much stronger than expected because we cannot find many counterexamples. The original publishers just went with the first, the article asserts the second.

corty 5y ago
I think both conclusions are wrong. Since this is a meta-study, drawing conclusions from the absence of null-results and the absence of clustering is prone to a lot of confounders. The article mentions publication-bias, where non-null-results are published with a higher probability. Other such confounders will exist, e.g. data-availability-bias in certain places and populations, location-bias because researchers study their usually first-world, urban, university-town surroundings more often. Community-bias, because researchers tend to cluster and get inspiration for new studies from their community and peer group on the conferences they visit. Data-similarity-bias, because they can only compare studies where data can be normalized to a common base. Quality-bias, because they necessarily include well-done and badly-done studies, in different amounts, usually without weighing them properly, because that is very hard to do. Most of those confounders cannot be corrected for. So one could draw a conclusion like "we do see null-results in the expected amount and clustering in the expected amount, therefore the result of this meta-study is a confirmation of the lead-crime-hypothesis". But the only correct opposite conclusion is "we do not see null-results and clustering in the expected amount, so our result is 'we do not know'". So the imho correct interpretation of the meta-study is "don't know, further research or better meta-analysis needed".
- ectopod 5y ago
  Don't forget outright fraud. https://news.ycombinator.com/item?id=27884233 https://news.ycombinator.com/item?id=27884233

paulpauper 5y ago

not really. the lead-crime hypothesis posits that the reduction of crime in the 90s is due to the reduction of lead, but if the effect is really tiny, which the paper suggests, then it likely means that other factors, such as increased incarceration and or accessibility to abortion, may also be to blame for falling crime, or something else entirely.