4 ms·

The title, while accurate, is perhaps unintentionally click-bait-y. The article includes a less obtuse alternate sub-title: "Is there a publication bias in beh

The title, while accurate, is perhaps unintentionally click-bait-y. The article includes a less obtuse alternate sub-title: "Is there a publication bias in behavioral intranasal oxytocin research on humans?" If I'd seen that I probably would not have clicked it to find out what on Earth a "file drawer problem" was.

deelowe 10y ago
The file drawer problem has been getting discussed quite a bit recently. There was an article that I read from month or so ago that specifically mentioned this being an issue across many fields of research with medical being the largest concern. There's a bit of a movement growing to try and address this throughout the scientific community. The tendency to only publish positive results has a chilling effect that could cause great harm to society.
- verytrivial 10y ago
  I agree, though I think the "file drawer problem" is still jargon and not quite a catch phrase (yet).
  - apathy 10y ago
    Publication bias is a substantially more descriptive and accurate phrase. Even when you do submit studies with negative results for publication, many journals simply will not accept them. Little to do with file drawers and much to do with editors. That said, power and experimental design -- can we confidently reject the hypothesis that an effect size of at least X is present between groups A and B (or models A and B) at some specified error bound? -- cannot be ignored. Bayesian or frequentist, at some point you have to conclude the experiment or write an interim report, and this involves making a decision in the face of uncertainty. Being a statistician means never having to say you're certain. It also means never being able to do so (unless you're an irresponsible asshole). You can declare a decision boundary and, based on that, evaluate the evidence. In light of this, it is important that negative studies not have inferior designs or sample sizes to the positive studies, and vice versa. In humans, it is exceedingly difficult to control for all of the factors you might want to. Nevertheless, the tendency of journals and PR outlets to favor sexy outliers over the weight of evidence is a huge source of this bias, which cannot be solely ascribed to unwillingness of researchers to submit the works for publication. (It is also why preprints are so valuable and their uptake so important.)
badloginagain 10y ago
"file drawer problem" Search Results: "The "file drawer problem" (a term coined in 1979 by Robert Rosenthal, a member of our Advisory Board) refers to the bias introduced into the scientific literature by selective publication--chiefly by a tendency to publish positive results but not to publish negative or nonconfirmatory results."
sctb 10y ago
Thanks, we've updated the submission title.

deelowe 10y ago

The file drawer problem has been getting discussed quite a bit recently. There was an article that I read from month or so ago that specifically mentioned this being an issue across many fields of research with medical being the largest concern. There's a bit of a movement growing to try and address this throughout the scientific community. The tendency to only publish positive results has a chilling effect that could cause great harm to society.

verytrivial 10y ago
I agree, though I think the "file drawer problem" is still jargon and not quite a catch phrase (yet).
- apathy 10y ago
  Publication bias is a substantially more descriptive and accurate phrase. Even when you do submit studies with negative results for publication, many journals simply will not accept them. Little to do with file drawers and much to do with editors. That said, power and experimental design -- can we confidently reject the hypothesis that an effect size of at least X is present between groups A and B (or models A and B) at some specified error bound? -- cannot be ignored. Bayesian or frequentist, at some point you have to conclude the experiment or write an interim report, and this involves making a decision in the face of uncertainty. Being a statistician means never having to say you're certain. It also means never being able to do so (unless you're an irresponsible asshole). You can declare a decision boundary and, based on that, evaluate the evidence. In light of this, it is important that negative studies not have inferior designs or sample sizes to the positive studies, and vice versa. In humans, it is exceedingly difficult to control for all of the factors you might want to. Nevertheless, the tendency of journals and PR outlets to favor sexy outliers over the weight of evidence is a huge source of this bias, which cannot be solely ascribed to unwillingness of researchers to submit the works for publication. (It is also why preprints are so valuable and their uptake so important.)

badloginagain 10y ago

"file drawer problem" Search Results: "The "file drawer problem" (a term coined in 1979 by Robert Rosenthal, a member of our Advisory Board) refers to the bias introduced into the scientific literature by selective publication--chiefly by a tendency to publish positive results but not to publish negative or nonconfirmatory results."

sctb 10y ago

Thanks, we've updated the submission title.