3 ms·

After spending time sysadmining in biotech and as a side-effect reading lots of papers in the field, my confidence in the validity of many scientific papers has

After spending time sysadmining in biotech and as a side-effect reading lots of papers in the field, my confidence in the validity of many scientific papers has shot way down. Coming from outside the scientific/academic arena, my view is that the system has become a resume/job padding tool for far too many people, who have simply learned how to A) hit all the main logic points the publishers/journals look for so they get published, and B) find ingenious ways to get their names attached to papers even if they were barely involved in at all. So tired of hearing X has been cited in over Y Z-journal papers! as a way of touting their scientific merit.

dj-wonk 11y ago
Right. How often do you hear people say "_ studies support my claim, _ studies controvert my claim, and _ studies are agnostic on my claim?" That would be more intellectually honest.
DanAndersen 11y ago
I suppose it's like any other metric, like software developers preening their GitHub activity logs so they always look active and constantly working). With so many PhDs and not enough positions, those in charge of hiring need ways to quantify the relative worth/ROI of applicants. This leads to a focus on things like impact factor, which correlate with but don't equate to what we actually care about. As with everything, you get what you measure. I'd like to learn more about the history of the Scientific Revolution, the Royal Society and all that. Probably the politics and human-nature is constant (Newton/Leibniz conflict is well-known), but I wonder if there was an impact on the quality of science by the fact that a lot of the research was done by old rich aristocrats whose livelihoods didn't depend on their scientific results -- although I guess their reputations did depend on it. Is that similar to or different from modern academics with tenure? Does the fact that there's a tenure selection process end up selecting good academics, or those who can work the system?

dj-wonk 11y ago

Right. How often do you hear people say "_ studies support my claim, _ studies controvert my claim, and _ studies are agnostic on my claim?" That would be more intellectually honest.

DanAndersen 11y ago

I suppose it's like any other metric, like software developers preening their GitHub activity logs so they always look active and constantly working). With so many PhDs and not enough positions, those in charge of hiring need ways to quantify the relative worth/ROI of applicants. This leads to a focus on things like impact factor, which correlate with but don't equate to what we actually care about. As with everything, you get what you measure. I'd like to learn more about the history of the Scientific Revolution, the Royal Society and all that. Probably the politics and human-nature is constant (Newton/Leibniz conflict is well-known), but I wonder if there was an impact on the quality of science by the fact that a lot of the research was done by old rich aristocrats whose livelihoods didn't depend on their scientific results -- although I guess their reputations did depend on it. Is that similar to or different from modern academics with tenure? Does the fact that there's a tenure selection process end up selecting good academics, or those who can work the system?