Ok, there is a real disagreement. As I understand it, reproducibility means "you can run this same experiment and expect to get the same effect." Reproducibili

Ok, there is a real disagreement. As I understand it, reproducibility means "you can run this same experiment and expect to get the same effect." Reproducibility is an issue because of study designs that are unusually to get accidental results. So you have your p value < .05, but the worry is that it's just an accident. If it's not an accident, then you should be able to run the exact same study again and get a similar effect. The problem of reproducibility is that even _that_ may not possible for some notable studies. Because of bad experimental design, we were just reporting meaningless results (the 5% chance that you get a nice p-value for an effect that isn't real). So in the case of this experiment, reproducibility is "if you survey people about bragging, will coders report their comments in the same way?" not "is bragging perceived negatively?" I think what you're describing is sometimes called a "conceptual reproduction" where you run a different study that you can argue supports the same conclusion. For instance, you've run an experiment where people seem to show a cognitive bias. Then someone else runs a different experiment, with a different design, and says "my results show the same cognitive bias that dbbolton showed in his study". In this case, the experimental design is different, and the second study doesn't directly demonstrate that you results could be reproduced. But in theory, it does suggest that there's a real phenomenon, because the same explanation seems natural for both experiments. There has been an overuse of conceptual reproductions in psychology, but what really has people worried is that major studies may literally not be reproducible. P.S. all of this is based on my understanding of how the terms are used, and I'm not 100% confident I'm right about it. I think I am, but I'm not a psychologist, take my opinion with a grain of salt.