5 ms·

Correct me if I'm wrong, but the average institution or entity that Bill Gates chooses to fund is much more productive than the average NIH funded lab. I think

by pen2l 7y ago

Correct me if I'm wrong, but the average institution or entity that Bill Gates chooses to fund is much more productive than the average NIH funded lab. I think the answer lies in taking a good look at where the money is going. Particularly, I think I think we need to give more to the very productive people and institutions. I know it's the million dollar question, by what is Salk doing that others are not? Or CSHL which has something like 8 nobel laureates there. One last observation is that a career in science just doesn't make sense anymore. Why slave away making $40 or $50k as a postdoc or an instructor after years of studious study when you can pull 6 figures in tech right out of school? This right here is one of the things that tells the bright ones to stay away from academia.

dnautics 7y ago
Ex-Scientist here: While I agree the gates foundation is likely doing a better job than the nih... The danger is this, how do you decide upon a metric for productivity? How do you then prevent 1) corruption, absent that, 2) inadvertent corruption via biases (I'm going to judge proposals in my field more favorably because they are less confusing or more comfortable, or the inverse, more harshly since I want to keep competition out or know the pitfalls better), and absent that, 3) goodhart's law?
- phkahler 7y ago
  Is there a psychological measure of internal bias? Hire decision makers that score low on that?
- mistermann 7y ago
  How about we vote on it?
- pdm55 7y ago
  Science is accidental, boring, repetitive, focused, finicky, wasted, political, ... A beast not easily tamed. Think of Edison's 100 trials to develop the light bulb. I continually think of my own truncated scientific career. I couldn't, early on in my projects, see the woods for the trees. About the only thing, I recognised was that I was just one cog in a vast enterprise. Looking back, I wish I had had the energy and insight to see beyond my lab work to where it might lead. Back in those days, you needed the term "business model" in your grant submission in order to attract funds. I walked away, thinking I had interests that could better generate a business.
- godelski 7y ago
  Current scientist: I think the answer is easy. Take more risk. Science is by nature risky. We're trying to figure out things that we don't know. That results in a lot of failure (I definitely wouldn't use the Silicon Valley model of failure being a metric of success though). One major change I would make from public money is publishing of results good or bad. There's this competition to get into top tier journals (which I don't think is bad or you can get rid of), but this is done at the cost of dumping good science just because it wasn't good enough for a top tier. Using impact as a metric is good but not great. The mark of a good scientist is their work and their passion, not where they publish. While there's a correlation, there's also Goodhart. If this is the only metric that we really concentrate on, then it will lead to slower science. We all know that we learn much more from failure than success. We all know that most of science is banging your head on the table for weeks out of frustration. But that's also not common knowledge. So I'd say that those funding science need to be more risky and those in science need to be more open about their failures and be realistic in their discussion of science.
  - sah2ed 7y ago
    > One major change I would make from public money is publishing of results good or bad. There's this competition to get into top tier journals (which I don't think is bad or you can get rid of), but this is done at the cost of dumping good science just because it wasn't good enough for a top tier. Using impact as a metric is good but not great. > The mark of a good scientist is their work and their passion, not where they publish. You are overlooking an invisible but equally important mark of a good scientist: a scientist’s impression on peers aka reputation, which usually comes down to two types: prestigious publication (in a prestigious journal or with a prestigious co-author) or having a good number of citations. The first kind of reputation is bequeathed almost immediately, the second kind of reputation is deferred — it can only be gained after the passage of time (years). If a scientist is known to have co-authored a paper that peers later on characterize as a seminal contribution to the field, peers are much more likely to pay attention to future papers from that scientist. The converse is true. The number of published papers, whether in prestigious journals or otherwise, far exceeds the number of people with free time and relevant qualifications to assess them. At the end of the day, human attention is a scarce resource, which is why it is the metric those aspiring to be regarded as “good scientists” in their field optimize for. Taken together, a scientist’s publication history is used as a proxy for authority in a field, which in turn is used to filter which papers to read or send to the trash can.
Retric 7y ago
The way to increase efficiency over time is to develop useful tools like CRISPR not simply look for the most beneficial research to solve a specific problem. Bill Gates has a focus on specific goals which are extremely valuable today, but arguably shortsighted. Further, the NIH is focused on solving problems for the UK not say Ethiopia. But, as poor countries develop they start using UK style solutions not cheaper though less effective alternatives. So it’s really a question of which metric you’re using to evaluate each system not some universal efficiency metric. Global impact over the next 20 years vs global impact over 200 years etc.
- wtvanhest 7y ago
  The older I have gotten, the more I believe that short term gains are better than long term strategies because short-term gains build on each other. I don’t know the history of crispr, but I doubt it was just a single check, single project completed from start to end. I’m sure it took a lot of iteration and success of many small short-term goals
  - Retric 7y ago
    It’s the nature of the goals not the time it takes to reach them that I am talking about. A COVID-19 vaccine is extremely valuable today, but it becomes useless even 5 years from now. On the other hand our experience with COVID-19 is likely to spawn a great deal of research into developing vaccines faster which while useless for this epidemic could be extremely valuable for the next several. Clearly a COVID-19 vaccine could save millions of lives and is worth investing in, but it’s a specific finite number. So called basic research can save more people’s lives than are alive today.
- runako 7y ago
  > develop useful tools like CRISPR not simply look for the most beneficial research to solve a specific problem You're describing a 40+ year body of research as if it were a project to develop CRISPR as an endpoint. But that's nothing like the process that actually produced CRISPR. Take the Hsu et. al. paper describing CRISPR-Cas9 (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4343198/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4343198/), for example. That paper cites 100 other papers, with multiple citations in each decade going back to the 1980s. (The first paper cited was published based on research that likely happened 10 years before before the sequencing of the entire genome of an organism.) And those 100 papers lean on other papers, etc. Science is not an engineering project.
lonelappde 7y ago
> Correct me if I'm wrong Can you start with any evidence? Correct me if I'm wrong: Gates projects are less productive than NIH, counterproductivey harming education policy, and focused on different, cheaper problems like non-US basic health and malaria.

by pen2l 7y ago

dnautics 7y ago

Ex-Scientist here: While I agree the gates foundation is likely doing a better job than the nih... The danger is this, how do you decide upon a metric for productivity? How do you then prevent 1) corruption, absent that, 2) inadvertent corruption via biases (I'm going to judge proposals in my field more favorably because they are less confusing or more comfortable, or the inverse, more harshly since I want to keep competition out or know the pitfalls better), and absent that, 3) goodhart's law?

phkahler 7y ago
Is there a psychological measure of internal bias? Hire decision makers that score low on that?
mistermann 7y ago
How about we vote on it?
pdm55 7y ago
Science is accidental, boring, repetitive, focused, finicky, wasted, political, ... A beast not easily tamed. Think of Edison's 100 trials to develop the light bulb. I continually think of my own truncated scientific career. I couldn't, early on in my projects, see the woods for the trees. About the only thing, I recognised was that I was just one cog in a vast enterprise. Looking back, I wish I had had the energy and insight to see beyond my lab work to where it might lead. Back in those days, you needed the term "business model" in your grant submission in order to attract funds. I walked away, thinking I had interests that could better generate a business.
godelski 7y ago
Current scientist: I think the answer is easy. Take more risk. Science is by nature risky. We're trying to figure out things that we don't know. That results in a lot of failure (I definitely wouldn't use the Silicon Valley model of failure being a metric of success though). One major change I would make from public money is publishing of results good or bad. There's this competition to get into top tier journals (which I don't think is bad or you can get rid of), but this is done at the cost of dumping good science just because it wasn't good enough for a top tier. Using impact as a metric is good but not great. The mark of a good scientist is their work and their passion, not where they publish. While there's a correlation, there's also Goodhart. If this is the only metric that we really concentrate on, then it will lead to slower science. We all know that we learn much more from failure than success. We all know that most of science is banging your head on the table for weeks out of frustration. But that's also not common knowledge. So I'd say that those funding science need to be more risky and those in science need to be more open about their failures and be realistic in their discussion of science.
- sah2ed 7y ago
  > One major change I would make from public money is publishing of results good or bad. There's this competition to get into top tier journals (which I don't think is bad or you can get rid of), but this is done at the cost of dumping good science just because it wasn't good enough for a top tier. Using impact as a metric is good but not great. > The mark of a good scientist is their work and their passion, not where they publish. You are overlooking an invisible but equally important mark of a good scientist: a scientist’s impression on peers aka reputation, which usually comes down to two types: prestigious publication (in a prestigious journal or with a prestigious co-author) or having a good number of citations. The first kind of reputation is bequeathed almost immediately, the second kind of reputation is deferred — it can only be gained after the passage of time (years). If a scientist is known to have co-authored a paper that peers later on characterize as a seminal contribution to the field, peers are much more likely to pay attention to future papers from that scientist. The converse is true. The number of published papers, whether in prestigious journals or otherwise, far exceeds the number of people with free time and relevant qualifications to assess them. At the end of the day, human attention is a scarce resource, which is why it is the metric those aspiring to be regarded as “good scientists” in their field optimize for. Taken together, a scientist’s publication history is used as a proxy for authority in a field, which in turn is used to filter which papers to read or send to the trash can.

Retric 7y ago

The way to increase efficiency over time is to develop useful tools like CRISPR not simply look for the most beneficial research to solve a specific problem. Bill Gates has a focus on specific goals which are extremely valuable today, but arguably shortsighted. Further, the NIH is focused on solving problems for the UK not say Ethiopia. But, as poor countries develop they start using UK style solutions not cheaper though less effective alternatives. So it’s really a question of which metric you’re using to evaluate each system not some universal efficiency metric. Global impact over the next 20 years vs global impact over 200 years etc.

wtvanhest 7y ago
The older I have gotten, the more I believe that short term gains are better than long term strategies because short-term gains build on each other. I don’t know the history of crispr, but I doubt it was just a single check, single project completed from start to end. I’m sure it took a lot of iteration and success of many small short-term goals
- Retric 7y ago
  It’s the nature of the goals not the time it takes to reach them that I am talking about. A COVID-19 vaccine is extremely valuable today, but it becomes useless even 5 years from now. On the other hand our experience with COVID-19 is likely to spawn a great deal of research into developing vaccines faster which while useless for this epidemic could be extremely valuable for the next several. Clearly a COVID-19 vaccine could save millions of lives and is worth investing in, but it’s a specific finite number. So called basic research can save more people’s lives than are alive today.
runako 7y ago
> develop useful tools like CRISPR not simply look for the most beneficial research to solve a specific problem You're describing a 40+ year body of research as if it were a project to develop CRISPR as an endpoint. But that's nothing like the process that actually produced CRISPR. Take the Hsu et. al. paper describing CRISPR-Cas9 (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4343198/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4343198/), for example. That paper cites 100 other papers, with multiple citations in each decade going back to the 1980s. (The first paper cited was published based on research that likely happened 10 years before before the sequencing of the entire genome of an organism.) And those 100 papers lean on other papers, etc. Science is not an engineering project.

lonelappde 7y ago

> Correct me if I'm wrong Can you start with any evidence? Correct me if I'm wrong: Gates projects are less productive than NIH, counterproductivey harming education policy, and focused on different, cheaper problems like non-US basic health and malaria.