14 ms·

Big Problems Facing Science

87 points by junipergreen 10y ago

hughw 10y ago
I had hoped to read one of those occasional lists of the hard, outstanding problems scientists still need to solve, in biology, say, or physics. Instead, it's a list of meta "problems" like science funding, or poor study designs.
- danieltillett 10y ago
  The only problem in science is funding. Everything else is an opportunity.
  - mac01021 10y ago
    And funding is not?
    - danieltillett 10y ago
      Well the general lack of funding and the way the limited funding is distributed is the ultimate cause of all the problems of science. I guess solving the funding problem is an opportunity, but it seems to be beyond us.
  - veddox 10y ago
    If we had unlimited funding, most of the other problems would indeed take care of themselves. The problem is the "unlimited" part...
    - danieltillett 10y ago
      The problem is not only the level of funding, but they way the funding is distributed. Science could live happily with the current expenditure if the way the funding was distributed was smarter.
- PeterWhittaker 10y ago
  I expected the "hard list" as well. And got one, just not the one I expected. There are multiple crises in science these days, whether of communication or recruitment or replication or of fraud or of anti-intellectualism fueled in part by the most egregious examples of these problems. Fixing these problems, addressing these seven, would do much to restore the lustre and respect and authority science well done and scientists working well deserve.
- whorleater 10y ago
  "Hard" problems can be eventually solved through tenacity, funding can't. Funnily enough, in my experience funding and proper experimental design is the hardest part of solving a problem.
- Strilanc 10y ago
  Wikipedia has a list for that: https://en.wikipedia.org/wiki/Lists_of_unsolved_problems https://en.wikipedia.org/wiki/Lists_of_unsolved_problems
drauh 10y ago
"Scientists often learn more from studies that fail. But failed studies can mean career death. So instead, they’re incentivized to generate positive results they can publish." Not in the "hard" sciences: my background is physics, and there are lots of papers on "failed studies", which serve to constrain the domain of applicability of some theory or other. Or, better yet, indicate new science to be found. The authors note the bias in the survey: "Our survey was not a scientific poll. For one, the respondents disproportionately hailed from the biomedical and social sciences and English-speaking communities."
- delazeur 10y ago
  To be fair, the science that the average person pays attention to generally falls within the biomedical and social sciences (e.g. studies about nutrition or how your job is making you depressed). Those are also the areas of science that have the biggest methodology problems, so from the perspective of a Vox writer it makes sense to talk about this as a "science" issue rather than a "biomedical and social science" issue.
- tagrun 10y ago
  As a physicist, I'm not happy with the trend of new research fields calling themselves science (mainly for benefiting from the hard-earned respect physics [or natural philosophy, as it was called] has gained throughout centuries, which is the real problem [rather than etymology]), and recently introduced concepts to gain some legitimacy, for justification, such as "hard science", "natural science", etc. https://en.wikipedia.org/wiki/Hard_and_soft_science#Criticism https://en.wikipedia.org/wiki/Hard_and_soft_science#Criticis... Now the word "science" somehow means "legitimate and respectable research". What's worse, it's not the reality of these fields but a distortion of reality through verbal association, and the word "science" is slowly being dragged into mud due to non-reproducible or downright wrong published results thanks to many such fields of "science". I should also add that mere "data fitting and data extrapolation" with no basic theory of fundamental understanding isn't science either. If you're curious about the details, Feynman has defined the issue very well at some point http://calteches.library.caltech.edu/51/2/CargoCult.pdf http://calteches.library.caltech.edu/51/2/CargoCult.pdf
veddox 10y ago
Yes, yes, yes - these are the things we ought to be talking about! In academia, in government, in society! Great to see these problems summarized here, usually one only sees an article on one of the seven. It gives me hope that we have started talking about these things - at least in academia (as numerous think-pieces in Nature et al. testify). We need to continue this discussion, make the public aware of it, and then start taking steps to solve it. No, science is not doomed, but boy do we still have a lot of work to do to get it to where it should be...
- themartorana 10y ago
  Yeah, what I came here to say. It's been interesting to watch over the past 5-10 years as these problems seemed to gain a voice. Hopefully we'll see several addressed in our lifetimes.
  - veddox 10y ago
    My favourite proposed solution for the paywall problem is to launch a platform akin to iTunes - one giant repository of academic articles available to anyone at a low cost. (Kind of like SciHub, except the people who deserve to get some money for their work actually do.)
- junipergreen 10y ago
  Yes, I think the general public sees science as an unstoppable, infallible force. Most scientists I know already know and talk about these problems (though I agree this is the most comprehensive, coherent presentation of them I've seen). I hope articles like this help non-scientists realize the that the institution of science is fragile, and must be fostered through innovation, funded, and actively protected.
stcredzero 10y ago
Could the "citizen science" movement help with the paywall problem? What about legislation requiring that any unclassified government funded science be available outside a paywall? Last I saw a figure for it, the combined budgets of all of the US military bands was ~$270 million a year. I bet a lot of science publication could be "nationalized" for that amount.
- veddox 10y ago
  The EU has put a regulation in place that as of 2020, all published research that funded by it must be open access (https://techcrunch.com/2016/05/27/eu-mandates-open-access-for-all-publicly-funded-research-by-2020/ https://techcrunch.com/2016/05/27/eu-mandates-open-access-fo...).
jaz46 10y ago
This list is great! Reproducibility and verification of results is key! Pachyderm (disclaimer: my company) is building infrastructure tools to help data scientists reproduce results by offering "Git for Data."
ap22213 10y ago
Science just costs too much, and scientists have paid too much to be considered scientists. The Ph.Ds that I know just want jobs that pay the bills, that pay off their loans. They do research because it is what gives them grants and enables the climb of the hierarchcy, not because they're driven by it. The system has beat out their lust of curiosity and exploration. The root problem is cost. We have to make science cheaper. We have to put it back in the hands of the curious and adventurous. Science should be possible by anyone - even teenagers. If science can't be done by the young, the poor, the autodidacts, what's the point? Currently, everyone is on the 'teach everyone programming' kick. U.S. states are now starting to require that everyone learns programming. But, what about science? Let's create the 'github' of science - where anyone with a hypothesis can create a notebook, gather up like-minded people to collaborate, gather data, analyze it, 'fork' others' research into new areas. That's how we will make it cheaper and accessible.
- thaw13579 10y ago
  Cost is an issue, but most labs already live "month-to-month" on grants and try to reduce the cost of research when possible.
- epicureanideal 10y ago
  What would be the "killer app" for such a platform?
- patall 10y ago
  No, that is not the problem. The science may be expensive but the main cost is seldomly the researcher him- or herself. Equipment is expensive and always will be, I mean we are speaking about cutting edge science not counting birds in the garden. When I am doing Assays in the lab, its it easily 5 bucks per sample. But then I have 30 to 50 samples. And one Assay is only a small step in a long procedure for a small scientific advance. Similar, when I am doing scientific computation, I often cannot work with the nice and cheap Nvidia 1080 GTX because I am doing science and need reproducibility. Accordingly, ECC is a must and we have to afford a Nvidia Tesla Cluster. Do you really think scientists are not curious or exploring, thats ridiculous. It is just that you cannot make an expedition without funding, and that is not about the money you make, its about the ship you need. (just my two cent as someone whos working on a dataset that is worth 3 or 4 times my yearly income and could easily be 3 or 4 (or 10) times larger)
d0mine 10y ago
It is an obvious propaganda piece for the benefit of those who profit on the fossil fuel industry. Psychology and Climate change are in different categories. The crisis of reproducibility in soft sciences has nothing to do with the overwhelming scientific consensus on the reality of global warming. Many mentioned issues are real in hard science too but it just an example that you can lie with the truth. Perverse incentives, publication bias, imperfection of peer review, etc can't invalidate established results e.g., Newtonian physics continues to work in the domain it is applicable for. It is infuriating that the planet (planetary habitability) is destroyed for the benefit of the very few.
- veddox 10y ago
  I'm sorry but I really cannot see where you get this bizarre interpretation of the article from. Global warming isn't mentioned anywhere in here! ("Climate change" gets exactly one mention, between two commas, and otherwise plays no role whatsoever.) I think you really missed the point of the article. Of course the problems the author talks about don't invalidate already established results, the big deal is that they potentially prevent new results from being soundly established (or discarded if necessary)!
  - d0mine 10y ago
    Are you a global warming denier? "Climate change" is "fossil fuel"-friendly alternative to avoid saying "Global warming" for what is happening. It is disingenuous to suggests that "Global warming" is unrelated to the article. The article equates the certainty with which we know results in Psychology and Climate change (the terms are separated by commas as you've noticed). Psychology has a very flimsy foundation: even major results can be debunked (e.g., ego depletion). On the other hand there is no doubt that the climate change (global warming) is happening. The article can be used as a tool by climate change deniers. They could say: "science have many major issues and therefore climate change is a figment of these communist eggheads imagination."
    - veddox 10y ago
      > Are you a global warming denier? I did not say anything to that effect in my previous comment. In fact, your question is so mindbogglingly groundless I am not even going to bother answering it. > The article can be used as a tool by climate change deniers. They could say: "science have many major issues and therefore climate change is a figment of these communist eggheads imagination." Of course it can, and nothing is going to stop them. But does that mean we shouldn't be talking about the problems we are facing in science? You aren't going to fix anything if you refuse to talk about it. This article isn't about climate change any more than it is about climate change deniers. That's another discussion for another time. This article is about our problems in the way we do science. So stick to the point, please.
junipergreen 10y ago
Daunting as this list seems, of these points and fixes are of course interrelated. Even small tweaking can bring big improvements. Take open science. Improvements to funding are needed to promote long-term projects, but in absence of those kinds of grants, open science is all the more important. Sharing partial results and details of studies that might not make it into publications leaves a baton for others to pick up and make the most of their limited funding time. And speaking of open science, making research public as it's happening (not even results, just what you're studying) can help prevent redundant concurrent studies in multiple labs and facilitate collaboration, also making the most of the limited funding scientists have.
jondubois 10y ago
I think that this trend of decreasing funding for science research will only continue unless some fundamental changes are made to the system. For one, the government is becoming increasingly weak while corporations are becoming increasingly powerful. Unlike the government, corporations in general (and their investment strategies) are focused on short term results - That's how executive pay/bonuses are structured - CEOs don't want to invest in something that will only bear fruit in 10 years so that some future CEO will get all the credit for it. Humans are terrible at allocating credit/praise because we like to pretend that the universe is simple and that all actions have simple, predictable effects without unexpected side-effects. Science research cannot exist in a corporate environment. Science can only rely on government or philanthropy.
unabst 10y ago
1. Money. Economical issue. 2. Poor design. Competence issue. 3. Replication. Incentive issue. 4. Peer review. Bad old analog system. 5. Paywalls. Capitalism. 6. Poor communication. Competence issue. 7. Stress. Personal issue. Nothing gets in the way of science except ourselves.

87 points by junipergreen 10y ago

hughw 10y ago

I had hoped to read one of those occasional lists of the hard, outstanding problems scientists still need to solve, in biology, say, or physics. Instead, it's a list of meta "problems" like science funding, or poor study designs.

danieltillett 10y ago
The only problem in science is funding. Everything else is an opportunity.
- mac01021 10y ago
  And funding is not?
  - danieltillett 10y ago
    Well the general lack of funding and the way the limited funding is distributed is the ultimate cause of all the problems of science. I guess solving the funding problem is an opportunity, but it seems to be beyond us.
- veddox 10y ago
  If we had unlimited funding, most of the other problems would indeed take care of themselves. The problem is the "unlimited" part...
  - danieltillett 10y ago
    The problem is not only the level of funding, but they way the funding is distributed. Science could live happily with the current expenditure if the way the funding was distributed was smarter.
PeterWhittaker 10y ago
I expected the "hard list" as well. And got one, just not the one I expected. There are multiple crises in science these days, whether of communication or recruitment or replication or of fraud or of anti-intellectualism fueled in part by the most egregious examples of these problems. Fixing these problems, addressing these seven, would do much to restore the lustre and respect and authority science well done and scientists working well deserve.
whorleater 10y ago
"Hard" problems can be eventually solved through tenacity, funding can't. Funnily enough, in my experience funding and proper experimental design is the hardest part of solving a problem.
Strilanc 10y ago
Wikipedia has a list for that: https://en.wikipedia.org/wiki/Lists_of_unsolved_problems https://en.wikipedia.org/wiki/Lists_of_unsolved_problems

drauh 10y ago

"Scientists often learn more from studies that fail. But failed studies can mean career death. So instead, they’re incentivized to generate positive results they can publish." Not in the "hard" sciences: my background is physics, and there are lots of papers on "failed studies", which serve to constrain the domain of applicability of some theory or other. Or, better yet, indicate new science to be found. The authors note the bias in the survey: "Our survey was not a scientific poll. For one, the respondents disproportionately hailed from the biomedical and social sciences and English-speaking communities."

delazeur 10y ago
To be fair, the science that the average person pays attention to generally falls within the biomedical and social sciences (e.g. studies about nutrition or how your job is making you depressed). Those are also the areas of science that have the biggest methodology problems, so from the perspective of a Vox writer it makes sense to talk about this as a "science" issue rather than a "biomedical and social science" issue.
tagrun 10y ago
As a physicist, I'm not happy with the trend of new research fields calling themselves science (mainly for benefiting from the hard-earned respect physics [or natural philosophy, as it was called] has gained throughout centuries, which is the real problem [rather than etymology]), and recently introduced concepts to gain some legitimacy, for justification, such as "hard science", "natural science", etc. https://en.wikipedia.org/wiki/Hard_and_soft_science#Criticism https://en.wikipedia.org/wiki/Hard_and_soft_science#Criticis... Now the word "science" somehow means "legitimate and respectable research". What's worse, it's not the reality of these fields but a distortion of reality through verbal association, and the word "science" is slowly being dragged into mud due to non-reproducible or downright wrong published results thanks to many such fields of "science". I should also add that mere "data fitting and data extrapolation" with no basic theory of fundamental understanding isn't science either. If you're curious about the details, Feynman has defined the issue very well at some point http://calteches.library.caltech.edu/51/2/CargoCult.pdf http://calteches.library.caltech.edu/51/2/CargoCult.pdf

veddox 10y ago

Yes, yes, yes - these are the things we ought to be talking about! In academia, in government, in society! Great to see these problems summarized here, usually one only sees an article on one of the seven. It gives me hope that we have started talking about these things - at least in academia (as numerous think-pieces in Nature et al. testify). We need to continue this discussion, make the public aware of it, and then start taking steps to solve it. No, science is not doomed, but boy do we still have a lot of work to do to get it to where it should be...

themartorana 10y ago
Yeah, what I came here to say. It's been interesting to watch over the past 5-10 years as these problems seemed to gain a voice. Hopefully we'll see several addressed in our lifetimes.
- veddox 10y ago
  My favourite proposed solution for the paywall problem is to launch a platform akin to iTunes - one giant repository of academic articles available to anyone at a low cost. (Kind of like SciHub, except the people who deserve to get some money for their work actually do.)
junipergreen 10y ago
Yes, I think the general public sees science as an unstoppable, infallible force. Most scientists I know already know and talk about these problems (though I agree this is the most comprehensive, coherent presentation of them I've seen). I hope articles like this help non-scientists realize the that the institution of science is fragile, and must be fostered through innovation, funded, and actively protected.

stcredzero 10y ago

Could the "citizen science" movement help with the paywall problem? What about legislation requiring that any unclassified government funded science be available outside a paywall? Last I saw a figure for it, the combined budgets of all of the US military bands was ~$270 million a year. I bet a lot of science publication could be "nationalized" for that amount.

veddox 10y ago
The EU has put a regulation in place that as of 2020, all published research that funded by it must be open access (https://techcrunch.com/2016/05/27/eu-mandates-open-access-for-all-publicly-funded-research-by-2020/ https://techcrunch.com/2016/05/27/eu-mandates-open-access-fo...).

jaz46 10y ago

This list is great! Reproducibility and verification of results is key! Pachyderm (disclaimer: my company) is building infrastructure tools to help data scientists reproduce results by offering "Git for Data."

ap22213 10y ago

Science just costs too much, and scientists have paid too much to be considered scientists. The Ph.Ds that I know just want jobs that pay the bills, that pay off their loans. They do research because it is what gives them grants and enables the climb of the hierarchcy, not because they're driven by it. The system has beat out their lust of curiosity and exploration. The root problem is cost. We have to make science cheaper. We have to put it back in the hands of the curious and adventurous. Science should be possible by anyone - even teenagers. If science can't be done by the young, the poor, the autodidacts, what's the point? Currently, everyone is on the 'teach everyone programming' kick. U.S. states are now starting to require that everyone learns programming. But, what about science? Let's create the 'github' of science - where anyone with a hypothesis can create a notebook, gather up like-minded people to collaborate, gather data, analyze it, 'fork' others' research into new areas. That's how we will make it cheaper and accessible.

thaw13579 10y ago
Cost is an issue, but most labs already live "month-to-month" on grants and try to reduce the cost of research when possible.
epicureanideal 10y ago
What would be the "killer app" for such a platform?
patall 10y ago
No, that is not the problem. The science may be expensive but the main cost is seldomly the researcher him- or herself. Equipment is expensive and always will be, I mean we are speaking about cutting edge science not counting birds in the garden. When I am doing Assays in the lab, its it easily 5 bucks per sample. But then I have 30 to 50 samples. And one Assay is only a small step in a long procedure for a small scientific advance. Similar, when I am doing scientific computation, I often cannot work with the nice and cheap Nvidia 1080 GTX because I am doing science and need reproducibility. Accordingly, ECC is a must and we have to afford a Nvidia Tesla Cluster. Do you really think scientists are not curious or exploring, thats ridiculous. It is just that you cannot make an expedition without funding, and that is not about the money you make, its about the ship you need. (just my two cent as someone whos working on a dataset that is worth 3 or 4 times my yearly income and could easily be 3 or 4 (or 10) times larger)

d0mine 10y ago

It is an obvious propaganda piece for the benefit of those who profit on the fossil fuel industry. Psychology and Climate change are in different categories. The crisis of reproducibility in soft sciences has nothing to do with the overwhelming scientific consensus on the reality of global warming. Many mentioned issues are real in hard science too but it just an example that you can lie with the truth. Perverse incentives, publication bias, imperfection of peer review, etc can't invalidate established results e.g., Newtonian physics continues to work in the domain it is applicable for. It is infuriating that the planet (planetary habitability) is destroyed for the benefit of the very few.

veddox 10y ago
I'm sorry but I really cannot see where you get this bizarre interpretation of the article from. Global warming isn't mentioned anywhere in here! ("Climate change" gets exactly one mention, between two commas, and otherwise plays no role whatsoever.) I think you really missed the point of the article. Of course the problems the author talks about don't invalidate already established results, the big deal is that they potentially prevent new results from being soundly established (or discarded if necessary)!
- d0mine 10y ago
  Are you a global warming denier? "Climate change" is "fossil fuel"-friendly alternative to avoid saying "Global warming" for what is happening. It is disingenuous to suggests that "Global warming" is unrelated to the article. The article equates the certainty with which we know results in Psychology and Climate change (the terms are separated by commas as you've noticed). Psychology has a very flimsy foundation: even major results can be debunked (e.g., ego depletion). On the other hand there is no doubt that the climate change (global warming) is happening. The article can be used as a tool by climate change deniers. They could say: "science have many major issues and therefore climate change is a figment of these communist eggheads imagination."
  - veddox 10y ago
    > Are you a global warming denier? I did not say anything to that effect in my previous comment. In fact, your question is so mindbogglingly groundless I am not even going to bother answering it. > The article can be used as a tool by climate change deniers. They could say: "science have many major issues and therefore climate change is a figment of these communist eggheads imagination." Of course it can, and nothing is going to stop them. But does that mean we shouldn't be talking about the problems we are facing in science? You aren't going to fix anything if you refuse to talk about it. This article isn't about climate change any more than it is about climate change deniers. That's another discussion for another time. This article is about our problems in the way we do science. So stick to the point, please.

junipergreen 10y ago

Daunting as this list seems, of these points and fixes are of course interrelated. Even small tweaking can bring big improvements. Take open science. Improvements to funding are needed to promote long-term projects, but in absence of those kinds of grants, open science is all the more important. Sharing partial results and details of studies that might not make it into publications leaves a baton for others to pick up and make the most of their limited funding time. And speaking of open science, making research public as it's happening (not even results, just what you're studying) can help prevent redundant concurrent studies in multiple labs and facilitate collaboration, also making the most of the limited funding scientists have.

jondubois 10y ago

I think that this trend of decreasing funding for science research will only continue unless some fundamental changes are made to the system. For one, the government is becoming increasingly weak while corporations are becoming increasingly powerful. Unlike the government, corporations in general (and their investment strategies) are focused on short term results - That's how executive pay/bonuses are structured - CEOs don't want to invest in something that will only bear fruit in 10 years so that some future CEO will get all the credit for it. Humans are terrible at allocating credit/praise because we like to pretend that the universe is simple and that all actions have simple, predictable effects without unexpected side-effects. Science research cannot exist in a corporate environment. Science can only rely on government or philanthropy.

unabst 10y ago

1. Money. Economical issue. 2. Poor design. Competence issue. 3. Replication. Incentive issue. 4. Peer review. Bad old analog system. 5. Paywalls. Capitalism. 6. Poor communication. Competence issue. 7. Stress. Personal issue. Nothing gets in the way of science except ourselves.