4 ms·

I am a post-doctoral research scientist that has wanted to be a scientist since early childhood (6-7 years old). The structure of how science is practiced seems

I am a post-doctoral research scientist that has wanted to be a scientist since early childhood (6-7 years old). The structure of how science is practiced seems to have changed substantially in that time frame, not for the better. Some of my childhood naivete plays a role in those perceptions, but many older scientists share that perspective. A lot of the problems in modern science revolve around the growing focus on bringing large quantities of research funds to your institution (which gets a large cut). Getting grants has an increasingly large need for political clout due to peer-review and increased competition due to reduced funding. Consequently, labs are getting larger and more hierarchical (i.e., an increased number of long-term post-docs and research scientists). As the organizational structure of science becomes less flat, the influence of those at the top is just reinforced and tends to constrain dialogue to fit established ideas. I think that the long-term progress of science depends on placing small bets on a greater variety of ideas rather than doubling down on fewer. Unfortunately, it will always be perceived to be safer to fund conventional ideas. Peer review enforces short-term, safe bet approaches. Large labs can churn out lots of papers, even if they are relatively financially inefficient. One way to correct this (if you agree that is a problem) might be to normalize grant scores by previous grant funding to the PI. i.e., (X papers of Y impact)/Z dollars of funding over the past 10 years. Double-blind review of grants might also help, but blinding is relatively easily circumvented.

dsp1234 11y ago
to normalize grant scores by previous grant funding to the PI. i.e., (X papers of Y impact)/Z dollars of funding over the past 10 years. One way to bypass this would be for larger organizations to internally incentivize their researchers to cite their own organizations papers in preference to, or in addition to, other organizations papers. Thus the larger the organization, the more it can press down on the PI scales. Alternately, even if it's not a several large organizations, smaller organizations can incentivize citing partner papers in the same way. Which is just restating the concept that if you measure something, and then base monetary rewards off that measurement, then that number will be hyper-optimized (even if it's not a good predictor of future performance). I don't have a good alternative, but just mentioning that there are easy circumventions (like you mentioned with double blinding)
entee 11y ago
I agree with most of what you're saying, but I think labs with more professional scientists (Research Scientists as opposed to postdocs and grad students) might be the way of the future and a Good Thing. There are obviously too many PhDs trained today to work in the academic market, but I think it's also likely we're training too many bioscience PhDs for industry as well. It would require more money for sure, but I think the concept of depending on an indentured servant class/apprenticeship model developed in the middle ages to execute most academic research isn't ideal. I think professionalizing the lower levels of academic science would make for a fairer, less exploitative, and because the practitioners would be more skilled, potentially more productive scientific academy.
danieltillett 11y ago
The problem is purely one of incentives. If you design a pathological system and make highly intelligent people use it then don’t be surprised with the terrible results. The solution to science is actually very simple - move from a peer review ranked grant allocation system (which is totally gamed by those at the top) to a basic screen and lottery system. The idea is to do a basic screen on grant applications to make sure they are scientifically viable and not majorly flawed (at least 75% of grants should pass this test) and then put them all into a pool and draw winners from this pool until you have allocated all the money you have. Lets stop using a system that can’t actually do the job (peer review can't separate the top 10% from the top 20%), and which is open to corruption and old boy networks, and move to one that is at least fair and better than all the alternatives.
- erikpukinskis 11y ago
  Or you could take the anarcho-syndicalist approach and split the money evenly to the entire pool, with a lower bound on grant size which makes the money inconsequential and if the lower limit is reached require all grantees to contribute a portion of their time to growing the money pool.
  - danieltillett 11y ago
    Two problem with this idea. If you don’t have some sort of barrier to entry then anyone can apply for a grant. This funding is made available on the basis that it will be used for science. I know this sounds elitist (it is), but science is really hard and it takes many years of learning to reach the forefront of knowledge where you can actually make a contribution. You only want to give funding to those that can actually use the money to do science. It would be a good idea to have another pool open to everyone to see if this barrier is really required - it might actually not be. The second problem is if you split the money too much then you won’t actually be able to do any research. To see an experiment through to completion requires a not insignificant amount of money (in most cases). If you start to hand out amounts of only a few thousand dollars at a time then nobody would be able to get any done.
daodedickinson 11y ago
My father definitely saw lab study fraud while getting his doctorate in the 70s. Academia in the US has resembled the game of thrones since at least the 60s campus revolutions.

dsp1234 11y ago

to normalize grant scores by previous grant funding to the PI. i.e., (X papers of Y impact)/Z dollars of funding over the past 10 years. One way to bypass this would be for larger organizations to internally incentivize their researchers to cite their own organizations papers in preference to, or in addition to, other organizations papers. Thus the larger the organization, the more it can press down on the PI scales. Alternately, even if it's not a several large organizations, smaller organizations can incentivize citing partner papers in the same way. Which is just restating the concept that if you measure something, and then base monetary rewards off that measurement, then that number will be hyper-optimized (even if it's not a good predictor of future performance). I don't have a good alternative, but just mentioning that there are easy circumventions (like you mentioned with double blinding)

entee 11y ago

I agree with most of what you're saying, but I think labs with more professional scientists (Research Scientists as opposed to postdocs and grad students) might be the way of the future and a Good Thing. There are obviously too many PhDs trained today to work in the academic market, but I think it's also likely we're training too many bioscience PhDs for industry as well. It would require more money for sure, but I think the concept of depending on an indentured servant class/apprenticeship model developed in the middle ages to execute most academic research isn't ideal. I think professionalizing the lower levels of academic science would make for a fairer, less exploitative, and because the practitioners would be more skilled, potentially more productive scientific academy.

danieltillett 11y ago

The problem is purely one of incentives. If you design a pathological system and make highly intelligent people use it then don’t be surprised with the terrible results. The solution to science is actually very simple - move from a peer review ranked grant allocation system (which is totally gamed by those at the top) to a basic screen and lottery system. The idea is to do a basic screen on grant applications to make sure they are scientifically viable and not majorly flawed (at least 75% of grants should pass this test) and then put them all into a pool and draw winners from this pool until you have allocated all the money you have. Lets stop using a system that can’t actually do the job (peer review can't separate the top 10% from the top 20%), and which is open to corruption and old boy networks, and move to one that is at least fair and better than all the alternatives.

erikpukinskis 11y ago
Or you could take the anarcho-syndicalist approach and split the money evenly to the entire pool, with a lower bound on grant size which makes the money inconsequential and if the lower limit is reached require all grantees to contribute a portion of their time to growing the money pool.
- danieltillett 11y ago
  Two problem with this idea. If you don’t have some sort of barrier to entry then anyone can apply for a grant. This funding is made available on the basis that it will be used for science. I know this sounds elitist (it is), but science is really hard and it takes many years of learning to reach the forefront of knowledge where you can actually make a contribution. You only want to give funding to those that can actually use the money to do science. It would be a good idea to have another pool open to everyone to see if this barrier is really required - it might actually not be. The second problem is if you split the money too much then you won’t actually be able to do any research. To see an experiment through to completion requires a not insignificant amount of money (in most cases). If you start to hand out amounts of only a few thousand dollars at a time then nobody would be able to get any done.

daodedickinson 11y ago

My father definitely saw lab study fraud while getting his doctorate in the 70s. Academia in the US has resembled the game of thrones since at least the 60s campus revolutions.