7 ms·

How life sciences work: Findings of a year-long investigation (2019)

56 points by johndcook 5y ago

strikingloo 5y ago
After this, Guzey went on to work on the NGO [New Science](https://newscience.org/ https://newscience.org/)
hammock 5y ago
I'm concerned by #3 "Nothing works the way you would naively think it works (for better and for worse)" which seems to imply that all or most of life sciences is funded by the NIH. What % of life sciences funding comes from the NIH and what are the other big sources?
- dekhn 5y ago
  In the US the NIH funds the vast majority of life sciences ($30B/year), but NSF, DOD(IIRC at least a billion a year each), and private institutions also provide life sciences funding (several more billions). NIH definitely is the big gorilla, many young investigators are trying to get established by getting R01s and using them to publish papers at the start of their career.
  - hammock 5y ago
    Fascinating. I guess I am surprised because I had thought of NIH as focused on human/public health, and there is so much to life sciences that is non-human. I would have thought there would be more major sources of funding. For example, something around ag science.
    - simplestats 5y ago
      The NSF funds basic science, including life sciences. NIH targets diseases. (and the NSF or NIH will not fund something if it better fits under the other) I certainly think it's a valuable thing to have a federal agency doing. If you want more basic life science, funding this area in the NSF more would be the way it's done in the system we have today. NIH has the cancer institute and the institute of mental health and so on. It is therefore really the domain of physicians. They want a path through mouse (or whatever) then human clinical studies, and want to see doctors as co-PI's. The fact that this careful practical system still pumps out so much bs is very problematic. I think it's the same problem as other research areas, not the fault of the NIH system itself.
  - panabee 5y ago
    the NIH is awesome. we owe those civil servants a debt of gratitude for sacrificing higher compensation in industry and performing as well as they do given the constraints. breakthrough ideas typically come from unconventional ideas, i.e., risky ideas or ideas likely to fail -- and thus waste taxpayer money. yet the funding process is mostly about conforming to traditional assumptions, which naturally gates scientific breakthroughs. the problem lies upstream. politicians don't allow the NIH to fund risky research and waste money.
    - trhway 5y ago
      >politicians don't allow the NIH to fund risky research how about NIH funding those gain-of-function coronavirus experiments (which included human testing) in Wuhan? And when DARPA refused to fund as too risky the additional human targeting coronavirus genetic manipulations in Wuhan, it was NIAID (led by Dr. Fauci - the top politician in life sciences today) which funded it.
      - hammock 5y ago
        >how about NIH funding those gain-of-function coronavirus experiments (which included human testing) in Wuhan? Politicians expressly disallowed those experiments. Don't stir stuff up just to stir. It was a non-political agency exec (Fauci?) who apparently sidestepped the typical review process where they would have been killed, and funded them anyway
        trhway 5y ago
        to me transfer of the research to China is exactly a typical solution by politicians - maintaining the perception of prohibition while in reality scaling the research even more into risky territory.
      - dekhn 5y ago
        you know what annoys me the most about this? I wanted to be an NIH researcher (US citizen, working in the US, on non-risky research and I couldn't get funding. That NIH is funding chinese institutes really bothers me, if legit us researchers can't get funding.
- ethanbond 5y ago
  The vast majority of life sciences is funded by NIH. This is one reason why the nonstop "revelations" of crazy projects funded by gasp Anthony Fauci are really meaningless. NIH doles out an enormous bucket of cash year after year for all sorts of research.
  - hammock 5y ago
    >the nonstop "revelations" of crazy projects funded by gasp Anthony Fauci are really meaningless I'm not really aware of any of those other than the gain of function research on bat coronaviruses by EcoHealth Alliance, which was only controversial beacause Fauci denied it in front of Congress and was technically not supposed to happen due to Obama's moratorium. What are the other revelations are you talking about?
    - dnautics 5y ago
      Not gp, but there was another one that made the rounds about dogs being subjected to being bitten by desert insects. Iirc It turned out that this wasn't funded by the nih?
- dnautics 5y ago
  Well for one, if anyone says "they weren't funded for X grant, so it didn't happen", it's implicitly encodes the idea that people don't do research before they get the grant money. They do.
- eganp 5y ago
  NIH likely funds 2/3rds of US biomedical research and ~1/3rd of global biomedical research by dollar value [0]. That link lists other big sources (ERC, MRC, US DoD, HHMI). My sense is that this landscape will change rather quickly with pandemic-inspired biomedical/defense initiatives and more private institutes in the US. Moreover, NIH funding, due to its central role in supporting biomedical research, is often necessary for getting a tenured PI position. Many high caliber R1 schools require multiple R01 NIH grants for tenure. Often private funding sources are less valuable to universities because they pay lower indirect rates (<10% vs >40%) on sponsored research. As institutions are so dependent on NIH funding, the NIH can exert influence far beyond the research it funds directly. [0] https://health-policy-systems.biomedcentral.com/articles/10.1186/s12961-015-0074-z https://health-policy-systems.biomedcentral.com/articles/10....
nosianu 5y ago
I looked at other articles on that author's site and found a topic that was discussed here more than once that I would be interested in hearing opinions about: https://guzey.com/books/why-we-sleep/ https://guzey.com/books/why-we-sleep/ The title is "Matthew Walker's "Why We Sleep" Is Riddled with Scientific and Factual Errors" and the contents is piece by piece refuting that book. Checking google, it was discussed on HN actually: https://news.ycombinator.com/item?id=21546850 https://news.ycombinator.com/item?id=21546850 and in this sub-thread: https://news.ycombinator.com/item?id=21792342 https://news.ycombinator.com/item?id=21792342 Quite a few other potentially interesting links, even if only to check some things more deeply. Also, https://guzey.com/fiction/hntop1/ https://guzey.com/fiction/hntop1/ ("How I got to #1 spot on Hacker News and why you should never try doing the same")
- ZhangSWEFAANG 5y ago
  I found his comments about giving up sex a little weird on the last article.
netizen-936824 5y ago
>I am confident that somewhere between 10% and 50% of papers published in good journals are wrong, meaningless or fraudulent. This is entirely unsurprising to me. On top of this, reporters will see something interesting in a big name journal and report on it yet that usually adds a whole 'nother layer of incorrectness that the general public picks up as "fact"
dnautics 5y ago
having been there (last was in the fray 7 years ago now... geeze time flies) here are my thoughts: 1. These are all technological developments, not scientific developments. Even MRNA-vaccines leading edge stuff was being built when I was in grad school (20 years ago). So judging by what technical developments are coming out now, may be a lagging indicator on science. 2-1: Absolutely correct. Out of 10 years in science I only did 1 "officially on what we were funded to do". The other nine were on sneakily independent shit, or on fully independent (hard money) experiments. 2-2: No comment either way, have no experience with 'methods development' at least in the sense that he's talking about. 2-3: correct in the short term but wrong in the long term. If you are foolish enough to be a postdoc that puts everything into the science, you won't get promoted, so your track record of being a good scientist ends on the vine. 3. Correct in the short term but wrong in the long. I did exactly this but couldn't keep going because being a postdoc was terminal for me. 4. Correct. I tick off at least four of these bullets, including "unwelcome demographics". 5-12. All correct, without much else to add. Well said! The biggest problem in sciences (biological) in the US is that we let the good quality scientists burn out and quit, and those who advance are mostly people who are playing the game (with a negative selection for those who are actually good scientists, because playing the game is so competitive at this point that it burns time effort and brainspace). This is a leadership defect. We don't give professors instructions on how to groom, including putting time, effort, and political capital into their grad students and postdocs to become lab leaders in their own right (if that is what they want) One thing that the author does not (and cannot) address is deep knowledge, or thinking about things from first principles, a la feynman or musk. I think as we have more and more interdisciplinary scientists jumping in at the interdisciplinary level, we're already in an era where diletanttery (especially in trendy science mashups like biophysics) is high. This will further dilute expertise and make it hard to separate the wheat from the lemons. Once had a coworker grad student think you could drop a bacterial plasmid into a mammalian cell and get protein expression. I told him I would do it for him, but "I did not think that would work". He then confidently reported a positive observation (which, thank god, did not make it into any sort of publication). Dude is now a associate professor of genome sciences at university of washington.
- analog31 5y ago
1vuio0pswjnm7 5y ago
"There were always large parts of science that were wrong or meaningless, so the present situation is not worse than the past in this respect. Even if 10-50% of the studies I mentioned in the first section turn out to be wrong, the pace of correct and important results is still astounding."
j7ake 5y ago
Nice article. I love hearing podcasts with scientists as guests (eg Boyden quote on expansion microscopy). Even better is when the hosts are themselves working scientists. Recently I got really into this podcast by Itai Yanai and Martin Lercher that tries to find how scientists navigate “day science” (eg formal papers and grants) vs “night science” (mucking about in the lab). Highly recommend if one wants to hear discussions by biological scientists talking about creativity. Check out “Night Science”.
- syspec 5y ago
  Any more recommendations?
  - j7ake 5y ago
    Unfortunately “Night science” is the only one I know where to the hosts are themselves established PIs. I imagine running a lab already takes 120% of ones time. I have no idea how Itai and Martin managed to do a podcast in addition.
j7ake 5y ago
There seems to be a tension between “all ideas come from PIs who direct slaves to do experiments” and “students and postdocs came up with their ideas and did everything, PI just paid salaries and equipment”. My experience is that all labs fall somewhere between these two extremes, and within labs people are spread along this axis. I felt a contradiction where the author states all ideas and execution comes from lab members, while quoting Boyden on the early days of expansion microscopy.
jostmey 5y ago
> I am confident that somewhere between 10% and 50% of papers published in good journals are wrong, meaningless or fraudulent. My experience is that this lies closer to 50%. This is the real problem with academic research and it’s only getting worse over time
- phgn 5y ago
  Just like the trend with big news media in general -- they understandably care about popularity, not facts. Why do we listen to those journals then, for the topics we care about?
peter303 5y ago
Sounds like a cult :-)

56 points by johndcook 5y ago

After this, Guzey went on to work on the NGO [New Science](https://newscience.org/ https://newscience.org/)

I'm concerned by #3 "Nothing works the way you would naively think it works (for better and for worse)" which seems to imply that all or most of life sciences is funded by the NIH. What % of life sciences funding comes from the NIH and what are the other big sources?

dekhn 5y ago
In the US the NIH funds the vast majority of life sciences ($30B/year), but NSF, DOD(IIRC at least a billion a year each), and private institutions also provide life sciences funding (several more billions). NIH definitely is the big gorilla, many young investigators are trying to get established by getting R01s and using them to publish papers at the start of their career.
- hammock 5y ago
  Fascinating. I guess I am surprised because I had thought of NIH as focused on human/public health, and there is so much to life sciences that is non-human. I would have thought there would be more major sources of funding. For example, something around ag science.
  - simplestats 5y ago
    The NSF funds basic science, including life sciences. NIH targets diseases. (and the NSF or NIH will not fund something if it better fits under the other) I certainly think it's a valuable thing to have a federal agency doing. If you want more basic life science, funding this area in the NSF more would be the way it's done in the system we have today. NIH has the cancer institute and the institute of mental health and so on. It is therefore really the domain of physicians. They want a path through mouse (or whatever) then human clinical studies, and want to see doctors as co-PI's. The fact that this careful practical system still pumps out so much bs is very problematic. I think it's the same problem as other research areas, not the fault of the NIH system itself.
- panabee 5y ago
  the NIH is awesome. we owe those civil servants a debt of gratitude for sacrificing higher compensation in industry and performing as well as they do given the constraints. breakthrough ideas typically come from unconventional ideas, i.e., risky ideas or ideas likely to fail -- and thus waste taxpayer money. yet the funding process is mostly about conforming to traditional assumptions, which naturally gates scientific breakthroughs. the problem lies upstream. politicians don't allow the NIH to fund risky research and waste money.
  - trhway 5y ago
    >politicians don't allow the NIH to fund risky research how about NIH funding those gain-of-function coronavirus experiments (which included human testing) in Wuhan? And when DARPA refused to fund as too risky the additional human targeting coronavirus genetic manipulations in Wuhan, it was NIAID (led by Dr. Fauci - the top politician in life sciences today) which funded it.
    - hammock 5y ago
      >how about NIH funding those gain-of-function coronavirus experiments (which included human testing) in Wuhan? Politicians expressly disallowed those experiments. Don't stir stuff up just to stir. It was a non-political agency exec (Fauci?) who apparently sidestepped the typical review process where they would have been killed, and funded them anyway
      - trhway 5y ago
        to me transfer of the research to China is exactly a typical solution by politicians - maintaining the perception of prohibition while in reality scaling the research even more into risky territory.
    - dekhn 5y ago
      you know what annoys me the most about this? I wanted to be an NIH researcher (US citizen, working in the US, on non-risky research and I couldn't get funding. That NIH is funding chinese institutes really bothers me, if legit us researchers can't get funding.
ethanbond 5y ago
The vast majority of life sciences is funded by NIH. This is one reason why the nonstop "revelations" of crazy projects funded by gasp Anthony Fauci are really meaningless. NIH doles out an enormous bucket of cash year after year for all sorts of research.
- hammock 5y ago
  >the nonstop "revelations" of crazy projects funded by gasp Anthony Fauci are really meaningless I'm not really aware of any of those other than the gain of function research on bat coronaviruses by EcoHealth Alliance, which was only controversial beacause Fauci denied it in front of Congress and was technically not supposed to happen due to Obama's moratorium. What are the other revelations are you talking about?
  - dnautics 5y ago
    Not gp, but there was another one that made the rounds about dogs being subjected to being bitten by desert insects. Iirc It turned out that this wasn't funded by the nih?
dnautics 5y ago
Well for one, if anyone says "they weren't funded for X grant, so it didn't happen", it's implicitly encodes the idea that people don't do research before they get the grant money. They do.
eganp 5y ago
NIH likely funds 2/3rds of US biomedical research and ~1/3rd of global biomedical research by dollar value [0]. That link lists other big sources (ERC, MRC, US DoD, HHMI). My sense is that this landscape will change rather quickly with pandemic-inspired biomedical/defense initiatives and more private institutes in the US. Moreover, NIH funding, due to its central role in supporting biomedical research, is often necessary for getting a tenured PI position. Many high caliber R1 schools require multiple R01 NIH grants for tenure. Often private funding sources are less valuable to universities because they pay lower indirect rates (<10% vs >40%) on sponsored research. As institutions are so dependent on NIH funding, the NIH can exert influence far beyond the research it funds directly. [0] https://health-policy-systems.biomedcentral.com/articles/10.1186/s12961-015-0074-z https://health-policy-systems.biomedcentral.com/articles/10....

nosianu 5y ago

I looked at other articles on that author's site and found a topic that was discussed here more than once that I would be interested in hearing opinions about: https://guzey.com/books/why-we-sleep/ https://guzey.com/books/why-we-sleep/ The title is "Matthew Walker's "Why We Sleep" Is Riddled with Scientific and Factual Errors" and the contents is piece by piece refuting that book. Checking google, it was discussed on HN actually: https://news.ycombinator.com/item?id=21546850 https://news.ycombinator.com/item?id=21546850 and in this sub-thread: https://news.ycombinator.com/item?id=21792342 https://news.ycombinator.com/item?id=21792342 Quite a few other potentially interesting links, even if only to check some things more deeply. Also, https://guzey.com/fiction/hntop1/ https://guzey.com/fiction/hntop1/ ("How I got to #1 spot on Hacker News and why you should never try doing the same")

ZhangSWEFAANG 5y ago
I found his comments about giving up sex a little weird on the last article.

netizen-936824 5y ago

>I am confident that somewhere between 10% and 50% of papers published in good journals are wrong, meaningless or fraudulent. This is entirely unsurprising to me. On top of this, reporters will see something interesting in a big name journal and report on it yet that usually adds a whole 'nother layer of incorrectness that the general public picks up as "fact"

dnautics 5y ago

having been there (last was in the fray 7 years ago now... geeze time flies) here are my thoughts: 1. These are all technological developments, not scientific developments. Even MRNA-vaccines leading edge stuff was being built when I was in grad school (20 years ago). So judging by what technical developments are coming out now, may be a lagging indicator on science. 2-1: Absolutely correct. Out of 10 years in science I only did 1 "officially on what we were funded to do". The other nine were on sneakily independent shit, or on fully independent (hard money) experiments. 2-2: No comment either way, have no experience with 'methods development' at least in the sense that he's talking about. 2-3: correct in the short term but wrong in the long term. If you are foolish enough to be a postdoc that puts everything into the science, you won't get promoted, so your track record of being a good scientist ends on the vine. 3. Correct in the short term but wrong in the long. I did exactly this but couldn't keep going because being a postdoc was terminal for me. 4. Correct. I tick off at least four of these bullets, including "unwelcome demographics". 5-12. All correct, without much else to add. Well said! The biggest problem in sciences (biological) in the US is that we let the good quality scientists burn out and quit, and those who advance are mostly people who are playing the game (with a negative selection for those who are actually good scientists, because playing the game is so competitive at this point that it burns time effort and brainspace). This is a leadership defect. We don't give professors instructions on how to groom, including putting time, effort, and political capital into their grad students and postdocs to become lab leaders in their own right (if that is what they want) One thing that the author does not (and cannot) address is deep knowledge, or thinking about things from first principles, a la feynman or musk. I think as we have more and more interdisciplinary scientists jumping in at the interdisciplinary level, we're already in an era where diletanttery (especially in trendy science mashups like biophysics) is high. This will further dilute expertise and make it hard to separate the wheat from the lemons. Once had a coworker grad student think you could drop a bacterial plasmid into a mammalian cell and get protein expression. I told him I would do it for him, but "I did not think that would work". He then confidently reported a positive observation (which, thank god, did not make it into any sort of publication). Dude is now a associate professor of genome sciences at university of washington.

analog31 5y ago

1vuio0pswjnm7 5y ago

"There were always large parts of science that were wrong or meaningless, so the present situation is not worse than the past in this respect. Even if 10-50% of the studies I mentioned in the first section turn out to be wrong, the pace of correct and important results is still astounding."

j7ake 5y ago

Nice article. I love hearing podcasts with scientists as guests (eg Boyden quote on expansion microscopy). Even better is when the hosts are themselves working scientists. Recently I got really into this podcast by Itai Yanai and Martin Lercher that tries to find how scientists navigate “day science” (eg formal papers and grants) vs “night science” (mucking about in the lab). Highly recommend if one wants to hear discussions by biological scientists talking about creativity. Check out “Night Science”.

syspec 5y ago
Any more recommendations?
- j7ake 5y ago
  Unfortunately “Night science” is the only one I know where to the hosts are themselves established PIs. I imagine running a lab already takes 120% of ones time. I have no idea how Itai and Martin managed to do a podcast in addition.

j7ake 5y ago

There seems to be a tension between “all ideas come from PIs who direct slaves to do experiments” and “students and postdocs came up with their ideas and did everything, PI just paid salaries and equipment”. My experience is that all labs fall somewhere between these two extremes, and within labs people are spread along this axis. I felt a contradiction where the author states all ideas and execution comes from lab members, while quoting Boyden on the early days of expansion microscopy.

jostmey 5y ago

> I am confident that somewhere between 10% and 50% of papers published in good journals are wrong, meaningless or fraudulent. My experience is that this lies closer to 50%. This is the real problem with academic research and it’s only getting worse over time

phgn 5y ago
Just like the trend with big news media in general -- they understandably care about popularity, not facts. Why do we listen to those journals then, for the topics we care about?

peter303 5y ago

Sounds like a cult :-)