3 ms·
Reading through the article, it seems the actual claim is much, much weaker than the title: > However, all intervention categories and domains apart from “fina
by malisper 4y ago
Reading through the article, it seems the actual claim is much, much weaker than the title:
> However, all intervention categories and domains apart from “finance” show evidence for heterogeneity, which implies that some nudges might be effective, even when there is evidence against the mean effect
So the article is saying when you look at studies of all "nudges" as whole, adjust for publication bias[0], there isn't evidence for nudges as a whole. Of course individual nudges could still have a positive impact.
Maybe I'm misinterpreting what it's saying, but as an analogy, that would be like doing a study of all "diets", determining that when you combine data for all studies on diets there isn't a positive effect, then writing an article with the claim "no evidence for diets". There's no way you could reasonably make the claim that no diets work.
[0] If you look at the studies on "nudges", studies with smaller sample sizes detected a larger positive effect size. This is because smaller studies that get a positive effect are more likely to be published than smaller studies with a negative effect. The article uses this to analyze just how strong the publication bias is and adjust for it.
- rahimnathwani 4y ago> There's no way you could reasonably make the claim that no diets work. Even if some studies results suggest that diets work, that does not, by itself, mean we should reject the null hypothesis (that diets don't work). Coincidences exist: https://xkcd.com/882/ https://xkcd.com/882/
- whimsicalism 4y agoYou are misinterpreting what it is saying. It's like multi comparison bias - if you do a bunch of comparisons and find one significant at 0.05, you can go ahead and say that is significant if you had looked at it alone. But if you consider all of the experiments you are doing, you would expect to find one as significant just by random chance even if it isn't significant.