This is extremely common in social science, for obvious reasons (imagine you're a social scientist and need to find 200 people to answer your questionnaire and

This is extremely common in social science, for obvious reasons (imagine you're a social scientist and need to find 200 people to answer your questionnaire and you don't have any money...), and I wouldn't be surprised if it has some part in the "replication crisis". A personal favourite example is that meme darling, the Dunning-Kruger effect. In addition to often being an extremely simplified version of the original proposition, which is a lot more subtle than "dumb people think they're smart", it's often presented as if (social!) science has uncovered a universal trait in the human race. But really this effect is based on very few studies, and they were mostly done on American college students. I wonder how well it replicates...