5 ms·

What is it that makes you have a nice career in research? Is it a robust pile of publishing or is it a star finding? Can you get far on just pure volume? I wan

What is it that makes you have a nice career in research? Is it a robust pile of publishing or is it a star finding? Can you get far on just pure volume? I want to answer the question "if I were a researcher and were willing to cheat to get ahead, what should be the objective of my cheating?"

lasfter 5y ago
I suppose it depends on how you define nice? If you cheat at some point people will catch on, even if you don't face any real consequences. So if you want prestige within your community then cheating isn't the way to go. If you want to look impressive to non-experts and get lots of grant money/opportunities, I'd go for lots of straightforward publications in top-tier venues. Star findings will come under greater scrutiny.
- rscho 5y ago
  Not outright cheating, but cooking results to seem better/surprising and publishing lots of those shitty papers is the optimal way to build a career in many fields. In medicine, for example.
  - lasfter 5y ago
    Cooking results seems like outright cheating to me.
    - DistressedDrone 5y ago
      It's more complicated than that. It can look something like this: https://xkcd.com/882/ https://xkcd.com/882/
gowld 5y ago
For grants and tenure, 100 tiny increments over 10 years are much better for your research career then 1 major paper in 5 years that is better than all of them put together. If you want to write a pop book and on TV and sell classes, you need one interesting bit of pseudoscience and a dozen followup papers using the same bad methodology.
- vngzs 5y ago
  This sounds inseparable from the replication crisis. The incentives are clearly broken: they are not structured in a manner that achieves the goal of research, which is to expand the scope and quality of human knowledge. To solve the crisis, we must change the incentives. Does anyone have ideas on how that may be achieved - what a correct incentive structure for research might look like?
  - PeterisP 5y ago
    I've thought about it a lot and I don't think it might be achieved. The trouble is that for the evaluators (all the institutions that can be sources of an incentive structure) it's impossible to distinguish an unpublished 90%-ready Nobel prize from unpublished 90%-ready bullshit. So if you've been working for 4 years on minor, incremental work and published a bunch of papers it's clear that you've done something useful, not extraordinary, but not bad; but if you've been working on a breakthrough and haven't achieved it, then there's simply no data to judge. Are you one step from major success? Or is that one step impossible and will never be achieved? Perhaps all of it is a dead end? Perhaps you're just slacking off on a direction that you know is a dead end, but it's the one thing you can do which brings you some money, so meh? Perhaps you're just crazy and it was definitely a worthless dead end? Perhaps everyone in the field thought that you're just crazy and this direction is worthless but they're actually wrong? Peter Higgs was a relevant case - IIRC he said in one interview taht for quite some time "they" didn't know what to do with him as he wasn't producing anything much, and the things he had done earlier were either useless or Nobel prize worthy, but it was impossible to tell for many years after the fact. How the heck can an objective incentive structure take that into account? It's a minefield. IMHO any effective solution has to scale back on accountability and measurability, and to some extent just give some funding to some people/teams with great potential, and see what they do - with the expectation that it's OK if it doesn't turn out, since otherwise they're forced to pick only safe topics that are certain to succeed and also certain to not achieve a breaktrhough. I believe European Research Foundation had a grant policy with similar principles, and I think that DARPA, at least originally, was like that. But there's a strong entirely opposite pressure from key stakeholders holding the (usually government) purses, their interests are more towards avoiding bad PR for any project with seemingly wasted money, and that results in a push towards these broken incentive structures and mediocrity.
caenn 5y ago
There's a book called science fictions that pretty much goes over the standard bullshit packages in modern science.
stewbrew 5y ago
You don't make a nice career in a vacuum. With very few exceptions, you don't get star findings in a social desert. You get star findings by being liked by influential supervisors who are liked by even more influential supervisors.
lelanthran 5y ago
> I want to answer the question "if I were a researcher and were willing to cheat to get ahead, what should be the objective of my cheating?" "Academic politics is the most vicious and bitter form of politics, because the stakes are so low." https://en.wikipedia.org/wiki/Sayre%27s_law https://en.wikipedia.org/wiki/Sayre%27s_law

lasfter 5y ago

I suppose it depends on how you define nice? If you cheat at some point people will catch on, even if you don't face any real consequences. So if you want prestige within your community then cheating isn't the way to go. If you want to look impressive to non-experts and get lots of grant money/opportunities, I'd go for lots of straightforward publications in top-tier venues. Star findings will come under greater scrutiny.

rscho 5y ago
Not outright cheating, but cooking results to seem better/surprising and publishing lots of those shitty papers is the optimal way to build a career in many fields. In medicine, for example.
- lasfter 5y ago
  Cooking results seems like outright cheating to me.
  - DistressedDrone 5y ago
    It's more complicated than that. It can look something like this: https://xkcd.com/882/ https://xkcd.com/882/

gowld 5y ago

For grants and tenure, 100 tiny increments over 10 years are much better for your research career then 1 major paper in 5 years that is better than all of them put together. If you want to write a pop book and on TV and sell classes, you need one interesting bit of pseudoscience and a dozen followup papers using the same bad methodology.

vngzs 5y ago
This sounds inseparable from the replication crisis. The incentives are clearly broken: they are not structured in a manner that achieves the goal of research, which is to expand the scope and quality of human knowledge. To solve the crisis, we must change the incentives. Does anyone have ideas on how that may be achieved - what a correct incentive structure for research might look like?
- PeterisP 5y ago
  I've thought about it a lot and I don't think it might be achieved. The trouble is that for the evaluators (all the institutions that can be sources of an incentive structure) it's impossible to distinguish an unpublished 90%-ready Nobel prize from unpublished 90%-ready bullshit. So if you've been working for 4 years on minor, incremental work and published a bunch of papers it's clear that you've done something useful, not extraordinary, but not bad; but if you've been working on a breakthrough and haven't achieved it, then there's simply no data to judge. Are you one step from major success? Or is that one step impossible and will never be achieved? Perhaps all of it is a dead end? Perhaps you're just slacking off on a direction that you know is a dead end, but it's the one thing you can do which brings you some money, so meh? Perhaps you're just crazy and it was definitely a worthless dead end? Perhaps everyone in the field thought that you're just crazy and this direction is worthless but they're actually wrong? Peter Higgs was a relevant case - IIRC he said in one interview taht for quite some time "they" didn't know what to do with him as he wasn't producing anything much, and the things he had done earlier were either useless or Nobel prize worthy, but it was impossible to tell for many years after the fact. How the heck can an objective incentive structure take that into account? It's a minefield. IMHO any effective solution has to scale back on accountability and measurability, and to some extent just give some funding to some people/teams with great potential, and see what they do - with the expectation that it's OK if it doesn't turn out, since otherwise they're forced to pick only safe topics that are certain to succeed and also certain to not achieve a breaktrhough. I believe European Research Foundation had a grant policy with similar principles, and I think that DARPA, at least originally, was like that. But there's a strong entirely opposite pressure from key stakeholders holding the (usually government) purses, their interests are more towards avoiding bad PR for any project with seemingly wasted money, and that results in a push towards these broken incentive structures and mediocrity.

caenn 5y ago

There's a book called science fictions that pretty much goes over the standard bullshit packages in modern science.

stewbrew 5y ago

You don't make a nice career in a vacuum. With very few exceptions, you don't get star findings in a social desert. You get star findings by being liked by influential supervisors who are liked by even more influential supervisors.

lelanthran 5y ago

> I want to answer the question "if I were a researcher and were willing to cheat to get ahead, what should be the objective of my cheating?" "Academic politics is the most vicious and bitter form of politics, because the stakes are so low." https://en.wikipedia.org/wiki/Sayre%27s_law https://en.wikipedia.org/wiki/Sayre%27s_law