5 ms·

-Nature is full of bogus, like all journals, if not more. The reason for its high impact factor isn't due to consistently solid papers all around, it's due to a

by aluren 8y ago

-Nature is full of bogus, like all journals, if not more. The reason for its high impact factor isn't due to consistently solid papers all around, it's due to a few very high impact, foundational papers and a crowd of minor impact/shaky ones that were sneaked in by big shots and friends of the editors. Look up retraction rates by journal and prepare to be surprised. -I don't think you realize the step it takes to actually write to Nature and say: "Excuse me, that paper you published is wrong and you should retract it." You're liable to alienate all of the authors as well as piss off the editorial board for pointing out that they let a mistake slip through. You should be prepared to face vigorous backlash, and be completely confident in your own results if you don't want the repercussions to overwhelm you. Many communities related to specific fields are small and niche, so making enemies with one team often means that you've actually cut yourself off a good part of that community. You should be prepared for awkward moments in conferences when you meet each other, incendiary questions when giving talks and scathing "anonymous" peer-reviews for your future submitted work. It's far easier to assume good faith, give the authors the benefit of the doubt ("yeah the implementation is buggy but what software isn't? The main idea is probably sound") and move on. -I don't know about other fields, but in mine, there's an unofficial accepted consensus on a set of very high-profile papers that happen to be either complete bunk or utterly useless. Again, we're talking about Science, Nature, etc. These papers made a lot of noise when they came out years ago, and now no one in their right mind would base their work on them. Again, you can't just knock on Science's editorial board and go "Excuse me, that hyped paper from a very big shot is useless and you shouldn't have published it", so it's just something people in the community know and whisper among themselves. I guess it may seem strange to outsiders though.

hnaccy 8y ago
> I guess it may seem strange to outsiders though. Of course it seems strange. The output of the scientific/academic community is often presented to people as being a source of truth and scientists do little to dissuade this. To learn that it's all rotten to the core is upsetting.
- aluren 8y ago
  It's not rotten to the core, it's just imperfect, like all human communities are. Nepotism, conflicts of interests, human error, unconscious biases, etc. don't magically disappear just because we're in academia. As I said below: "All in all, I'd say we're doing fine. We're just not the ethereal source of truth that some people hold us to be, the very same people who, after claming that 'God is dead', are very quick to replace Him with His sillicon-based equivalent around which we would act like priests, except with lab coats in lieu of clerical garments." It's all about trust, really. Many people choose to trust state-of-the-art medical research when they do vaccinations on their kids; some do not. You can indeed choose not to trust us as a 'source of truth', and it seems a fair chunk of the US political landscape is being led down that path; we'll see where it goes.
  - lvh 8y ago
    Most of those other systems don’t claim to be designed to prevent them intrinsically via process, though.
    - aluren 8y ago
      We never claimed to be completely free of bias, or immune to nepotism. We're human. I think you're making it to be a bigger deal than it actually is. Most of these things are just noise. We just ignore it. If someone rises to prominence with outrageously fake results, rest assured that they will get shot down quickly by concurrents (all the incentive to publish fake results lies within highly competitive fields). If you know what you're doing, and work in a lab where people know what they're doing too, you can still do pretty good science. I would guess than in all communities, academic or not, there's a nonzero amount of bogus, over-hype, nepotism, politics and what not, and a subset of people who actually get things done and make the whole field advance. After a while, the test of time truly determines what was actually useful from what is bunk. That's how it's always been.
      - cwyers 8y ago
        Brian Wansink didn't get shot down quickly at all.
feral 8y ago
Pretty interesting use of 'assume good faith' there! > people in the community know and whisper among themselves If no one else, tenured academics should be calling out bunk in their fields, or are failing, IMO.
deleted 8y ago
[deleted]
ecocentrik 8y ago
> You're liable to alienate all of the authors as well as piss off the editorial board for pointing out that they let a mistake slip through. Better rewards for uncovering bad science keeps it from devolving into a political contest. If you're not seeing the right incentives in place, you're either missing something or you're seeing an opportunity to capture some value by implementing those incentives.
- chithanh 8y ago
  > Better rewards for uncovering bad science keeps it from devolving into a political contest. No. If you increase rewards for uncovering bad science, it will just lead to false accusations. > If you're not seeing the right incentives in place Some people think that the very existence of incentives is the problem here. https://hbr.org/1993/09/why-incentive-plans-cannot-work https://hbr.org/1993/09/why-incentive-plans-cannot-work
  - ecocentrik 8y ago
    > If you increase rewards for uncovering bad science, it will just lead to false accusations. Isn't that what this study suggests is already happening, in as much as false accusations are just more bad science? > Some people think that the very existence of incentives is the problem here. https://hbr.org/1993/09/why-incentive-plans-cannot-work https://hbr.org/1993/09/why-incentive-plans-cannot-work This article is full of opaque generalizations about human behavior that more accurately describes the baser impulses of individuals attempting to game systems they don't understand (participants in psychological experiments) and don't so much reflect the actions of self-conscious professionals with any semblance of dedication to their fields.
ajnin 8y ago
The role of science in society is to increase the sum of knowledge on which progress can be made. If a scientist knowingly lets false information be published he is causing a disservice to society, harming progress. Scientific papers are made public for a reason. The correct information should not be accessible only to a small circle of people in the know. If incorrect papers are out there, those who can correct them have a duty to do so.
- aluren 8y ago
  >If a scientist knowingly lets false information be published he is causing a disservice to society, harming progress. Often it's not so much false as empty or useless. Nevertheless, since all people in a community know the values of individual papers, and these are the same people 'making progress' in that field, I don't see how science is so much harmed. I agree the situation is kinda ridiculous, but it's not a "sky is falling" situation either. >If incorrect papers are out there, those who can correct them have a duty to do so. Sure. How much are you willing to pay me for me to prioritize this 'duty' over the literal hundreds of duties I already have at my lab?
YeGoblynQueenne 8y ago
Curiosity killed the cat and all that, but what is your field? I had a look at your profile, comments and submissions (hopefully that doesn't sound creepy!) and I can't tell. It's Saturday morning (and I got students' papers to mark) :)

by aluren 8y ago

hnaccy 8y ago

> I guess it may seem strange to outsiders though. Of course it seems strange. The output of the scientific/academic community is often presented to people as being a source of truth and scientists do little to dissuade this. To learn that it's all rotten to the core is upsetting.

aluren 8y ago
It's not rotten to the core, it's just imperfect, like all human communities are. Nepotism, conflicts of interests, human error, unconscious biases, etc. don't magically disappear just because we're in academia. As I said below: "All in all, I'd say we're doing fine. We're just not the ethereal source of truth that some people hold us to be, the very same people who, after claming that 'God is dead', are very quick to replace Him with His sillicon-based equivalent around which we would act like priests, except with lab coats in lieu of clerical garments." It's all about trust, really. Many people choose to trust state-of-the-art medical research when they do vaccinations on their kids; some do not. You can indeed choose not to trust us as a 'source of truth', and it seems a fair chunk of the US political landscape is being led down that path; we'll see where it goes.
- lvh 8y ago
  Most of those other systems don’t claim to be designed to prevent them intrinsically via process, though.
  - aluren 8y ago
    We never claimed to be completely free of bias, or immune to nepotism. We're human. I think you're making it to be a bigger deal than it actually is. Most of these things are just noise. We just ignore it. If someone rises to prominence with outrageously fake results, rest assured that they will get shot down quickly by concurrents (all the incentive to publish fake results lies within highly competitive fields). If you know what you're doing, and work in a lab where people know what they're doing too, you can still do pretty good science. I would guess than in all communities, academic or not, there's a nonzero amount of bogus, over-hype, nepotism, politics and what not, and a subset of people who actually get things done and make the whole field advance. After a while, the test of time truly determines what was actually useful from what is bunk. That's how it's always been.
    - cwyers 8y ago
      Brian Wansink didn't get shot down quickly at all.

feral 8y ago

Pretty interesting use of 'assume good faith' there! > people in the community know and whisper among themselves If no one else, tenured academics should be calling out bunk in their fields, or are failing, IMO.

deleted 8y ago

[deleted]

ecocentrik 8y ago

> You're liable to alienate all of the authors as well as piss off the editorial board for pointing out that they let a mistake slip through. Better rewards for uncovering bad science keeps it from devolving into a political contest. If you're not seeing the right incentives in place, you're either missing something or you're seeing an opportunity to capture some value by implementing those incentives.

chithanh 8y ago
> Better rewards for uncovering bad science keeps it from devolving into a political contest. No. If you increase rewards for uncovering bad science, it will just lead to false accusations. > If you're not seeing the right incentives in place Some people think that the very existence of incentives is the problem here. https://hbr.org/1993/09/why-incentive-plans-cannot-work https://hbr.org/1993/09/why-incentive-plans-cannot-work
- ecocentrik 8y ago
  > If you increase rewards for uncovering bad science, it will just lead to false accusations. Isn't that what this study suggests is already happening, in as much as false accusations are just more bad science? > Some people think that the very existence of incentives is the problem here. https://hbr.org/1993/09/why-incentive-plans-cannot-work https://hbr.org/1993/09/why-incentive-plans-cannot-work This article is full of opaque generalizations about human behavior that more accurately describes the baser impulses of individuals attempting to game systems they don't understand (participants in psychological experiments) and don't so much reflect the actions of self-conscious professionals with any semblance of dedication to their fields.

ajnin 8y ago

The role of science in society is to increase the sum of knowledge on which progress can be made. If a scientist knowingly lets false information be published he is causing a disservice to society, harming progress. Scientific papers are made public for a reason. The correct information should not be accessible only to a small circle of people in the know. If incorrect papers are out there, those who can correct them have a duty to do so.

aluren 8y ago
>If a scientist knowingly lets false information be published he is causing a disservice to society, harming progress. Often it's not so much false as empty or useless. Nevertheless, since all people in a community know the values of individual papers, and these are the same people 'making progress' in that field, I don't see how science is so much harmed. I agree the situation is kinda ridiculous, but it's not a "sky is falling" situation either. >If incorrect papers are out there, those who can correct them have a duty to do so. Sure. How much are you willing to pay me for me to prioritize this 'duty' over the literal hundreds of duties I already have at my lab?

YeGoblynQueenne 8y ago

Curiosity killed the cat and all that, but what is your field? I had a look at your profile, comments and submissions (hopefully that doesn't sound creepy!) and I can't tell. It's Saturday morning (and I got students' papers to mark) :)