5 ms·

In a lot of fields, you just can't do this. We have to remember that a single article should not be taken as gospel, but more as a step in a direction, like a s

by Balgair 10y ago

In a lot of fields, you just can't do this. We have to remember that a single article should not be taken as gospel, but more as a step in a direction, like a single log entry in a captain's journal. "We saw X happen Y many times in a 1000+Z step process with ABC cells in FGH media at P temperature at N phase of the moon. Our p value is being required by the journal, but is meaningless because we are just reporting that you should look out for this thing, if it exists at all"

btilly 10y ago
In a lot of fields you can. In fact an entire field (ego depletion) seems to have been ENTIRELY produced by this type of publication selection bias.
- BeetleB 10y ago
  Interesting - I did not know that. Looking at the criticism section in the Wikipedia article, your claim is as suspect as ego depletion :-) It seems the jury is still out.
  - btilly 10y ago
    My selecting that as an example is based on http://www.slate.com/articles/health_and_science/cover_story/2016/03/ego_depletion_an_influential_theory_in_psychology_may_have_just_been_debunked.html http://www.slate.com/articles/health_and_science/cover_story.... My phrasing "seems to have been" reflects both my opinion, and the fact that it is far from certain. That said, regardless of what happens with this specific theory, it is well worth reading Richard Feynman's remarks from 42 years ago in http://calteches.library.caltech.edu/51/2/CargoCult.htm http://calteches.library.caltech.edu/51/2/CargoCult.htm. Until a field of research can demonstrate the kind of integrity that Feynman talks about there, nobody should believe what they claim. They can talk all that they want about how much work they did, the publication history, the supporting evidence - it is all BS until you have that. Increasingly people call themselves "scientists" and call what they are doing "science" while lacking that fundamental foundation. The result is that it is becoming harder and harder to tell which claims about reality deserve provisional belief versus healthy skepticism.

by Balgair 10y ago

btilly 10y ago

In a lot of fields you can. In fact an entire field (ego depletion) seems to have been ENTIRELY produced by this type of publication selection bias.

BeetleB 10y ago
Interesting - I did not know that. Looking at the criticism section in the Wikipedia article, your claim is as suspect as ego depletion :-) It seems the jury is still out.
- btilly 10y ago
  My selecting that as an example is based on http://www.slate.com/articles/health_and_science/cover_story/2016/03/ego_depletion_an_influential_theory_in_psychology_may_have_just_been_debunked.html http://www.slate.com/articles/health_and_science/cover_story.... My phrasing "seems to have been" reflects both my opinion, and the fact that it is far from certain. That said, regardless of what happens with this specific theory, it is well worth reading Richard Feynman's remarks from 42 years ago in http://calteches.library.caltech.edu/51/2/CargoCult.htm http://calteches.library.caltech.edu/51/2/CargoCult.htm. Until a field of research can demonstrate the kind of integrity that Feynman talks about there, nobody should believe what they claim. They can talk all that they want about how much work they did, the publication history, the supporting evidence - it is all BS until you have that. Increasingly people call themselves "scientists" and call what they are doing "science" while lacking that fundamental foundation. The result is that it is becoming harder and harder to tell which claims about reality deserve provisional belief versus healthy skepticism.