4 ms·

It's not necessarily even status, just feeding themselves: Publish or die, and you aren't publishing if your conclusions aren't positive and strong. In medicin

It's not necessarily even status, just feeding themselves: Publish or die, and you aren't publishing if your conclusions aren't positive and strong. In medicine it's especially tough, as testing is among the most expensive and the number of available data points is quite low, so one gets to fignt over individual data points that make a difference. Did this person not follow the protocol, and I have a paper, or not? I've seen something similar when working in agriculture: Early tests of new plant strains are low data, because a company will start by testing so many experimental plants that it'd be unaffordable to test them all very strongly. This makes people really argue about single data points in those early tests. Was this chunk of the field contaminated? Attacked by a wild animal? But there at least the interest in fraud is small. If a breeder gets a stinker through, it just goes into trials on the other hemisphere, where it's planted in an order of magnitude more fields, and therefore just lead to disappointment 6 months later. With medicine, the cost of replication is so high, and often takes so many years, that it's not just that an honest mistake is catastrophic, but that the difference in outcomes for the person doing the curation is so high, one doesn't have to be all that dishonest to make biased decisions that will lead to a strong career.

BurningFrog 3y ago
> It's not necessarily even status, just feeding themselves: Publish or die, and you aren't publishing if your conclusions aren't positive and strong. The wider science needs to learn a lesson from the Economics science: Incentives Matter. A lot.
- jampekka 3y ago
  Much of the problem is that wider science has taken too many lessons from economics. Incentives matter a lot but creating incentives that work is almost impossible. Science used to run on ethics (like many other professions). But we learned from economists that there are no such thing, only utility (money) maximizers.
  - nvm0n2 3y ago
    Incentives that work are easy. Do research in corporate labs. That's all it takes. Now there is an incentive to do work that replicates (because it is intended to be used to build useful things), there are people responsible for detecting and resolving fraud (managers), they are incentivized to do so by a mix of carrots and sticks some of them legal. Fact is, corporate R&D doesn't have this relentless problem with reproducibility. It's academic output that does, because academics only care about getting a paper published and don't expect that anyone will use their results. Often they don't even make their code or data available at all because it's not to their advantage for others to be able to replicate their work, as that would yield fewer papers. But this is all wrong. Science exists to be used in technology, not for its own sake.
    - jampekka 3y ago
      There are corresponding responsibilities in academia. Perhaps even more so. But like a business culture can, and often does, rot the incentives and responsibilities, so has the academic culture. I've worked for two corporate labs and have collaborated with several. There was less rigor if anything. Corporate lab work is quite cozy and stable. In corporate lab your job doesn't end if you don't (pretend) to get a new major discovery every year. They don't make anything available, often not even internally. Science is about a lot more than technology.
      - BurningFrog 3y ago
        > like a business culture can, and often does, rot the incentives and responsibilities, so has the academic culture. Maybe the problem is that there are many corporations, with different cultures. But there is mostly just one academic culture across the whole system.
      - nvm0n2 3y ago
        Hmm I've had the opposite experiences. Corp labs release cool and useful stuff all the time. On the front page right now is Seamless, a very useful thing released by a corporate lab. And corps have performance evaluation and management programmes designed to encourage high performance output. And of course the competent ones manage to bring research into production regularly. Whereas with academics, you get a paper. You might get data and code, or might not, depending on field and temperament of the researchers. If you're really lucky that data/code might actually be correct, match the paper and be usable for something, but it really depends a lot on the field. You almost certainly won't get products.

BurningFrog 3y ago

> It's not necessarily even status, just feeding themselves: Publish or die, and you aren't publishing if your conclusions aren't positive and strong. The wider science needs to learn a lesson from the Economics science: Incentives Matter. A lot.

jampekka 3y ago
Much of the problem is that wider science has taken too many lessons from economics. Incentives matter a lot but creating incentives that work is almost impossible. Science used to run on ethics (like many other professions). But we learned from economists that there are no such thing, only utility (money) maximizers.
- nvm0n2 3y ago
  Incentives that work are easy. Do research in corporate labs. That's all it takes. Now there is an incentive to do work that replicates (because it is intended to be used to build useful things), there are people responsible for detecting and resolving fraud (managers), they are incentivized to do so by a mix of carrots and sticks some of them legal. Fact is, corporate R&D doesn't have this relentless problem with reproducibility. It's academic output that does, because academics only care about getting a paper published and don't expect that anyone will use their results. Often they don't even make their code or data available at all because it's not to their advantage for others to be able to replicate their work, as that would yield fewer papers. But this is all wrong. Science exists to be used in technology, not for its own sake.
  - jampekka 3y ago
    There are corresponding responsibilities in academia. Perhaps even more so. But like a business culture can, and often does, rot the incentives and responsibilities, so has the academic culture. I've worked for two corporate labs and have collaborated with several. There was less rigor if anything. Corporate lab work is quite cozy and stable. In corporate lab your job doesn't end if you don't (pretend) to get a new major discovery every year. They don't make anything available, often not even internally. Science is about a lot more than technology.
    - BurningFrog 3y ago
      > like a business culture can, and often does, rot the incentives and responsibilities, so has the academic culture. Maybe the problem is that there are many corporations, with different cultures. But there is mostly just one academic culture across the whole system.
    - nvm0n2 3y ago
      Hmm I've had the opposite experiences. Corp labs release cool and useful stuff all the time. On the front page right now is Seamless, a very useful thing released by a corporate lab. And corps have performance evaluation and management programmes designed to encourage high performance output. And of course the competent ones manage to bring research into production regularly. Whereas with academics, you get a paper. You might get data and code, or might not, depending on field and temperament of the researchers. If you're really lucky that data/code might actually be correct, match the paper and be usable for something, but it really depends a lot on the field. You almost certainly won't get products.