4 ms·

It's a little annoying if they're cherry-picking but does it really impact the validity of the results? If anything, I'd think they'd have a greater incentive t

• exolymph 8y ago
  Yes, p-hacking and related techniques do impact the validity of the results. More on that here: https://info.umkc.edu/drbanderson/p-hacking-and-the-problem-of-multiple-comparisons/ https://info.umkc.edu/drbanderson/p-hacking-and-the-problem-...
• darkpuma 8y ago
  > "It's a little annoying if they're cherry-picking but does it really impact the validity of the results?" Yes, that's a well known method of p-hacking.
• samatman 8y ago
  The Relevant Xkcd is actually an excellent guide to why this is a problem: https://xkcd.com/882/ https://xkcd.com/882/
• pergadad 8y ago
  "I want to prove that it's good for something, but I don't care what it's good for" is not really a good research question. Then you'll measure 100 things and only publish the one that makes your intervention look good and possibly ignore all the ones that make it look bad. It's also rather useless if you don't at least have some idea of the mechanism. Coffee has lots of different parts, so what part of it is useful - is it the caffeine? The tannins? ...? And how does it have this effect? Is it even the coffee or just that the coffee group took 5 minutes of rest to drink their brew and that is really the effect?