15 ms·

A grant proposal shouldn’t hinge on whether research is driven by a hypothesis

44 points by digital55 9y ago

sorokod 9y ago
Newton was looking for the Philosopher's Stone. Why shouldn't we? https://en.wikipedia.org/wiki/Isaac_Newton's_occult_studies https://en.wikipedia.org/wiki/Isaac_Newton's_occult_studies
- maxerickson 9y ago
  The fun thing is that modern science has transmuted other elements into gold.
  - sorokod 9y ago
    Not quite there with regards to immortality (a property of lapis philosophorum) though :-)
    - deleted 9y ago
      [deleted]
    - Retra 9y ago
      We did find some clues, though.
    - danbruc 9y ago
      You probably don't want to become immortal. At first glance it may sound desirable, but once you start actually thinking about it, you quickly realize that you were probably pretty naive about that. You may however want to live a bit longer than what your current life expectation grants you.
deleted 9y ago
[deleted]
rococode 9y ago
Has using a quote from 4 centuries ago to try to prove a point ever worked out for anyone? It sounds about as effective as "Newton didn't use vaccines, so why should we?". Perhaps there is some merit to the idea, but talking about 17th century science is probably not the most convincing way to make the point. That aside, here is Newton's actual quote from Cohen and Whitman's translation of Principia: > I have not as yet been able to discover the reason for these properties of gravity from phenomena, and I do not feign hypotheses. For whatever is not deduced from the phenomena must be called a hypothesis; and hypotheses, whether metaphysical or physical, or based on occult qualities, or mechanical, have no place in experimental philosophy. In this philosophy particular propositions are inferred from the phenomena, and afterwards rendered general by induction. [1] To me, this appears to be a very different sentiment. Newton seems to be saying "I don't try to guess about things I don't understand", not "You don't need to have a clear plan before doing research". Even if it is true that Newton never framed a hypothesis in his life (which strikes me as blatantly false), he didn't live in a world with thousands and thousands of researchers funded by taxpayer money. When commitees have to decide how to spend money, of course it's more logical to fund a project with a well-formed question. [1] https://en.wikipedia.org/wiki/Hypotheses_non_fingo https://en.wikipedia.org/wiki/Hypotheses_non_fingo
- mannykannot 9y ago
  A little further into the article, the author presents many other arguments in support of his point, and discusses why some types of experiment (e.g., typical medical investigations) may need hypotheses, while others need not (the Hubble telescope was not built to investigate a specific hypothesis.) This article deserves more of a response than one that ignores anything not mentioned in the title.
  - gowld 9y ago
    I'm reminded of research in Computer Science, that generally looks like this: "In this paper we investigate the hypothesis: Is it possible to build something cool? First, we briefly summarize cool things that already exist, then we show how we were able to build something cool, we measure how cool it is (it's cooler than a bunch of competing things!), and finally conclude with some cool ideas that we or you might pursue the future."
SubiculumCode 9y ago
In neuroimaging research, I often would rather think of myself as a kind of explorer/cartographer, just weakly mapping the brain in order to fuel future strong hypothesis testing. That is one reason why I think opening and sharing our data is so fundamental to the lasting value of our work.
ChuckMcM 9y ago
The article answers its own question with this bit -- "Yet grant-writing experts universally stress that proposals should be built around hypotheses and warn that those not written this way risk rejection as “fishing expeditions.” "Fishing Expedition" is code for a project that spends a bunch of money and has no tangible result. Typically that is called wasted money. When you have a patron that wants you go out and follow your curiosity, where ever it might lead, that is great. But if you're spending the tax payer's dollar, the folks who gave it to you have to explain to the congressional budget office why they gave it to you and what they hoped to achieve by giving it to you. The easiest way to express that value proposition is with a hypothesis statement. Building equipment that is known to work (like the Hubble) doesn't require a hypothesis, it requires a credible statement that it will be able to do something that other equipment cannot. Or that you could not build the same capability in a different more cost effective way.
- gowld 9y ago
  The Space Program has CBO-friendly reason for existing (beat the Russkies to the moon), but 99% of its value was unrelated to that reason. AT&T invented huge swaths of technology motivated simply by something valuable would come own of free-range research. PhDs and tenured professors do the same, usually when their research area happens to be inexpensive. Maybe flimflamming and petty rivalries are not the best justification for basic science?
  - twblalock 9y ago
    > Maybe flimflamming and petty rivalries are not the best justification for basic science? Getting value for taxpayer money is not flimflamming, and it's not petty. There are a lot of good things to spend that money on, and if scientists want a piece of it they should provide a convincing justification.
    - ThomPete 9y ago
      Basic research in hard sciences is a justification in itself. Maybe drop doing phds in social sciences and humanities and use only for hard sciences is getting value for taxpayer money?
zck 9y ago
When taking science classes in school, I never understood why a hypothesis had to take a side. It could not be a question, like "hypothesis: does hot water freeze faster than cold water?", but had to take a position on the issue: "hypothesis: hot water freezes faster than cold water". I never got why this was the case; it sets the experiment up for failure: "oh no, I proved my hypothesis wrong". Who cares? As long as you show something (even not reflecting the null hypothesis is _something_) with a well-designed experiment, why does it matter if your guess is right or not? I'd much rather prove a hypothesis like "a chalk circle drawn around sugar does nothing to inhibit ants" wrong, than to prove it right! That would be much more interesting!
- dragonwriter 9y ago
  > When taking science classes in school, I never understood why a hypothesis had to take a side. It could not be a question, like "hypothesis: does hot water freeze faster than cold water?", Because a hypothesis is a falsifiable answer, not a question. > I never got why this was the case; it sets the experiment up for failure: Setting up an explanation for falsification is exactly why. But falsification isn't failure of the experiment, only the hypothesis.
- automatoney 9y ago
  I'm not sure what level of schooling you're talking about, but I think there's two reasons to take a side. First, I think in early school it prepares you to think in a way that makes sense with null/alternative hypotheses and all of those z and t stats hypothesis tests. The second reason is to have you think about what you expect to happen. If you're doing an experiment, you likely have some information to come up with what you think will happen, and what the explanation for that is. If you get a result that goes against what your reasoning was beforehand, then you have a misconception and have to adjust what you think you know. If you don't take a side, then you could potentially say that you believed that would happen all along, even if it doesn't make sense with how you understand things. Taking a side makes you confront your confusion when it doesn't work out as you thought.
  - analog31 9y ago
deleted 9y ago
[deleted]
pradn 9y ago
It seems like the idealist model of science - observable data, falsifiable hypothesis, repeatable experiment - is often not how science works. Aeon has a good article on this subject. https://aeon.co/essays/a-fetish-for-falsification-and-observation-holds-back-science https://aeon.co/essays/a-fetish-for-falsification-and-observ...
forkandwait 9y ago
https://plato.stanford.edu/entries/feyerabend/ https://plato.stanford.edu/entries/feyerabend/
deleted 9y ago
[deleted]
cbhl 9y ago
Scientific funding is actually one of the things I would want to cut to fund a UBI. I feel like a lot of "college professor/researcher" work is dealing with the paperwork of justifying why someone should give you enough money that you can pay an undergrad or master's student enough money to cover half of tuition so that you can get them to do some experiments with you. If people didn't have to worry about rent or putting food on the table, they wouldn't have to spend time on government paperwork and could just focus on advancing science instead.
- saagarjha 9y ago
  Are you suggesting that scientific funding doesn’t have benefits for society, or that it’s benefits are outweighed by the need for UBI?
  - cbhl 9y ago
    Neither -- I am saying that scientific funding primarily serves to keep scientists housed and alive (okay, and a little equipment too) and the money would go further without the overhead of means testing (that is, squabbling over which research is "worth funding" and which is not).
  - bonniemuffin 9y ago
    I think the poster is suggesting that if people didn't have to work for money, many scientists would continue doing scientific research for free, and they'd have more time to spend on it because they wouldn't have to play the grants game. This might actually be true for a field like math, where you just need paper/computer to make a major discovery. Probably not for a field like biology that involves a lot of web lab materials -- paying the humans is only a tiny sliver of the cost of making a discovery in the biosciences.
- newen 8y ago
  People who do science aren't some weird platonic ideal of scientists who are only motivated by science. They have spouses, kids they want to send to college, friends, social reputations, and a mortgage. I don't know any scientist who is willing to live in a crappy house (which is highly likely with only UBI) and crappy salary so that they can do science 100% of the time.
ajross 9y ago
It's actually not a terrible article. But argh is the framing in the title awful. Fine, fine. Isaac ?!@# Newton can get away with theorizing ab initio and turn out useful theories. You can't. More to the point, and relevant to the practice of modern science and not the Work of Giants: experiments designed without an eye to a clear hypothesis are pretty much guaranteed to be exercises in p-hacking.
- dang 9y ago
  Ok, let's try the subtitle.
throwawy43018 9y ago
And then there's the Broad Institute[1]. For some research a hypothesis is too reductionist. Sometimes you need to catalog at exhaustive scale: sequence the entire human genome, catalogue the structure of all cell types[5], and their pathways[6], log all gene expression for every tissue type in as many states of disease as possible[3], examine repurposing drugs by exposing every type of cell to every compound in the US Pharmacopoeia to look for novel effects (via transcriptomics)[2] Genomic sequencing, microfluidic liquid handling[4,7], and novel statistical approaches allow for scientific knowledge to be generated at an astounding pace. After the data exist, then we can make hypotheses. This is the new model for biomedical big science. This model of science is often criticized for not being hypothesis-driven, either as "stamp collecting," or as a force for vacuuming up funding that would otherwise support myriad small labs. It does bear fruit for schizophrenia[8], depression [9], as well as diabetes, obesity, and heart disease [10]. Where are the most interesting, most complex, and most challenging data problems these days? Not advertising. Not finance. Not crypto. Biology. (And many from cryptography, math, CS, and physics have switched gears to work on biological problems.) We run on messy stochastic software, and we finally have the tools to read the source code and save edits. It's an exciting time. 1. https://www.broadinstitute.org/about-us https://www.broadinstitute.org/about-us 2. https://clue.io/repurposing https://clue.io/repurposing 3. https://www.gtexportal.org/home/ https://www.gtexportal.org/home/ 4. https://www.broadinstitute.org/news/dronc-seq-microfluidic-single-nucleus-rna-sequencing https://www.broadinstitute.org/news/dronc-seq-microfluidic-s... 5. https://www.broadinstitute.org/news/international-human-cell-atlas-initiative-gets-underway https://www.broadinstitute.org/news/international-human-cell... 6. https://www.broadinstitute.org/klarman-cell-observatory/approach https://www.broadinstitute.org/klarman-cell-observatory/appr... 7. https://www.broadinstitute.org/files/patents/WO2016149661.pdf https://www.broadinstitute.org/files/patents/WO2016149661.pd... 8. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4112379/pdf/emss-59304.pdf https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4112379/pdf/ems... 9. https://www.nature.com/articles/s41588-018-0090-3 https://www.nature.com/articles/s41588-018-0090-3 10. (Via profile of common author in several fields) https://scholar.google.com/citations?hl=en&user=EISUuucAAAAJ&view_op=list_works https://scholar.google.com/citations?hl=en&user=EISUuucAAAAJ...
fizixer 9y ago
> The success of a grant proposal shouldn’t hinge on whether the research is driven by a hypothesis, especially in the physical sciences. You're looking at it the wrong way. If one call for proposal (meaning availability to fund only one project) receives a hundred proposals, then the acceptance rate is going to be 1% no matter what. Requirements like hypothesis inclusion are just ways to split the weed out work into two phases, e.g., eliminate 50 of the 100 proposals because they don't have hypothesis, and then eliminate 49 out of the remaining 50 after more careful reading. If you don't have the hypothesis requirement, that only means 99 out of 100 are eliminated after full reading. And what do you think the likelihood is going to be that the reviewers would accept your hypothesis-free proposal over dozens others that have one (all other things considered equal)? The problem is lack of funding (or alternatively, too many researchers and not enough research work to go around, depending on how you look at it; Masters degree is the new Bachelors degree, PhD is the new Masters, etc, etc). What you're focusing on are marginal issues.
firasd 9y ago
I finally 'got' the concept of hypotheses after a couple stupid public policy experiments here in India. The 'Odd-Even' car rationing experiment in Delhi to reduce pollution was praised because it reduced traffic. Then the country-wide demonetization disaster meant to reduce undeclared income was later touted as a way to enable digital transactions. So to generalize, if you're not clear on what you're trying to achieve, you can claim success by retrofitting your results to any stated purpose.

44 points by digital55 9y ago

Newton was looking for the Philosopher's Stone. Why shouldn't we? https://en.wikipedia.org/wiki/Isaac_Newton's_occult_studies https://en.wikipedia.org/wiki/Isaac_Newton's_occult_studies

maxerickson 9y ago
The fun thing is that modern science has transmuted other elements into gold.
- sorokod 9y ago
  Not quite there with regards to immortality (a property of lapis philosophorum) though :-)
  - deleted 9y ago
    [deleted]
  - Retra 9y ago
    We did find some clues, though.
  - danbruc 9y ago
    You probably don't want to become immortal. At first glance it may sound desirable, but once you start actually thinking about it, you quickly realize that you were probably pretty naive about that. You may however want to live a bit longer than what your current life expectation grants you.

deleted 9y ago

[deleted]

rococode 9y ago

Has using a quote from 4 centuries ago to try to prove a point ever worked out for anyone? It sounds about as effective as "Newton didn't use vaccines, so why should we?". Perhaps there is some merit to the idea, but talking about 17th century science is probably not the most convincing way to make the point. That aside, here is Newton's actual quote from Cohen and Whitman's translation of Principia: > I have not as yet been able to discover the reason for these properties of gravity from phenomena, and I do not feign hypotheses. For whatever is not deduced from the phenomena must be called a hypothesis; and hypotheses, whether metaphysical or physical, or based on occult qualities, or mechanical, have no place in experimental philosophy. In this philosophy particular propositions are inferred from the phenomena, and afterwards rendered general by induction. [1] To me, this appears to be a very different sentiment. Newton seems to be saying "I don't try to guess about things I don't understand", not "You don't need to have a clear plan before doing research". Even if it is true that Newton never framed a hypothesis in his life (which strikes me as blatantly false), he didn't live in a world with thousands and thousands of researchers funded by taxpayer money. When commitees have to decide how to spend money, of course it's more logical to fund a project with a well-formed question. [1] https://en.wikipedia.org/wiki/Hypotheses_non_fingo https://en.wikipedia.org/wiki/Hypotheses_non_fingo

mannykannot 9y ago
A little further into the article, the author presents many other arguments in support of his point, and discusses why some types of experiment (e.g., typical medical investigations) may need hypotheses, while others need not (the Hubble telescope was not built to investigate a specific hypothesis.) This article deserves more of a response than one that ignores anything not mentioned in the title.
- gowld 9y ago
  I'm reminded of research in Computer Science, that generally looks like this: "In this paper we investigate the hypothesis: Is it possible to build something cool? First, we briefly summarize cool things that already exist, then we show how we were able to build something cool, we measure how cool it is (it's cooler than a bunch of competing things!), and finally conclude with some cool ideas that we or you might pursue the future."

SubiculumCode 9y ago

In neuroimaging research, I often would rather think of myself as a kind of explorer/cartographer, just weakly mapping the brain in order to fuel future strong hypothesis testing. That is one reason why I think opening and sharing our data is so fundamental to the lasting value of our work.

ChuckMcM 9y ago

The article answers its own question with this bit -- "Yet grant-writing experts universally stress that proposals should be built around hypotheses and warn that those not written this way risk rejection as “fishing expeditions.” "Fishing Expedition" is code for a project that spends a bunch of money and has no tangible result. Typically that is called wasted money. When you have a patron that wants you go out and follow your curiosity, where ever it might lead, that is great. But if you're spending the tax payer's dollar, the folks who gave it to you have to explain to the congressional budget office why they gave it to you and what they hoped to achieve by giving it to you. The easiest way to express that value proposition is with a hypothesis statement. Building equipment that is known to work (like the Hubble) doesn't require a hypothesis, it requires a credible statement that it will be able to do something that other equipment cannot. Or that you could not build the same capability in a different more cost effective way.

gowld 9y ago
The Space Program has CBO-friendly reason for existing (beat the Russkies to the moon), but 99% of its value was unrelated to that reason. AT&T invented huge swaths of technology motivated simply by something valuable would come own of free-range research. PhDs and tenured professors do the same, usually when their research area happens to be inexpensive. Maybe flimflamming and petty rivalries are not the best justification for basic science?
- twblalock 9y ago
  > Maybe flimflamming and petty rivalries are not the best justification for basic science? Getting value for taxpayer money is not flimflamming, and it's not petty. There are a lot of good things to spend that money on, and if scientists want a piece of it they should provide a convincing justification.
  - ThomPete 9y ago
    Basic research in hard sciences is a justification in itself. Maybe drop doing phds in social sciences and humanities and use only for hard sciences is getting value for taxpayer money?

zck 9y ago

When taking science classes in school, I never understood why a hypothesis had to take a side. It could not be a question, like "hypothesis: does hot water freeze faster than cold water?", but had to take a position on the issue: "hypothesis: hot water freezes faster than cold water". I never got why this was the case; it sets the experiment up for failure: "oh no, I proved my hypothesis wrong". Who cares? As long as you show something (even not reflecting the null hypothesis is _something_) with a well-designed experiment, why does it matter if your guess is right or not? I'd much rather prove a hypothesis like "a chalk circle drawn around sugar does nothing to inhibit ants" wrong, than to prove it right! That would be much more interesting!

dragonwriter 9y ago
> When taking science classes in school, I never understood why a hypothesis had to take a side. It could not be a question, like "hypothesis: does hot water freeze faster than cold water?", Because a hypothesis is a falsifiable answer, not a question. > I never got why this was the case; it sets the experiment up for failure: Setting up an explanation for falsification is exactly why. But falsification isn't failure of the experiment, only the hypothesis.
automatoney 9y ago
I'm not sure what level of schooling you're talking about, but I think there's two reasons to take a side. First, I think in early school it prepares you to think in a way that makes sense with null/alternative hypotheses and all of those z and t stats hypothesis tests. The second reason is to have you think about what you expect to happen. If you're doing an experiment, you likely have some information to come up with what you think will happen, and what the explanation for that is. If you get a result that goes against what your reasoning was beforehand, then you have a misconception and have to adjust what you think you know. If you don't take a side, then you could potentially say that you believed that would happen all along, even if it doesn't make sense with how you understand things. Taking a side makes you confront your confusion when it doesn't work out as you thought.
- analog31 9y ago

deleted 9y ago

[deleted]

pradn 9y ago

It seems like the idealist model of science - observable data, falsifiable hypothesis, repeatable experiment - is often not how science works. Aeon has a good article on this subject. https://aeon.co/essays/a-fetish-for-falsification-and-observation-holds-back-science https://aeon.co/essays/a-fetish-for-falsification-and-observ...

forkandwait 9y ago

https://plato.stanford.edu/entries/feyerabend/ https://plato.stanford.edu/entries/feyerabend/

deleted 9y ago

[deleted]

cbhl 9y ago

Scientific funding is actually one of the things I would want to cut to fund a UBI. I feel like a lot of "college professor/researcher" work is dealing with the paperwork of justifying why someone should give you enough money that you can pay an undergrad or master's student enough money to cover half of tuition so that you can get them to do some experiments with you. If people didn't have to worry about rent or putting food on the table, they wouldn't have to spend time on government paperwork and could just focus on advancing science instead.

saagarjha 9y ago
Are you suggesting that scientific funding doesn’t have benefits for society, or that it’s benefits are outweighed by the need for UBI?
- cbhl 9y ago
  Neither -- I am saying that scientific funding primarily serves to keep scientists housed and alive (okay, and a little equipment too) and the money would go further without the overhead of means testing (that is, squabbling over which research is "worth funding" and which is not).
- bonniemuffin 9y ago
  I think the poster is suggesting that if people didn't have to work for money, many scientists would continue doing scientific research for free, and they'd have more time to spend on it because they wouldn't have to play the grants game. This might actually be true for a field like math, where you just need paper/computer to make a major discovery. Probably not for a field like biology that involves a lot of web lab materials -- paying the humans is only a tiny sliver of the cost of making a discovery in the biosciences.
newen 8y ago
People who do science aren't some weird platonic ideal of scientists who are only motivated by science. They have spouses, kids they want to send to college, friends, social reputations, and a mortgage. I don't know any scientist who is willing to live in a crappy house (which is highly likely with only UBI) and crappy salary so that they can do science 100% of the time.

ajross 9y ago

It's actually not a terrible article. But argh is the framing in the title awful. Fine, fine. Isaac ?!@# Newton can get away with theorizing ab initio and turn out useful theories. You can't. More to the point, and relevant to the practice of modern science and not the Work of Giants: experiments designed without an eye to a clear hypothesis are pretty much guaranteed to be exercises in p-hacking.

dang 9y ago
Ok, let's try the subtitle.

throwawy43018 9y ago

And then there's the Broad Institute[1]. For some research a hypothesis is too reductionist. Sometimes you need to catalog at exhaustive scale: sequence the entire human genome, catalogue the structure of all cell types[5], and their pathways[6], log all gene expression for every tissue type in as many states of disease as possible[3], examine repurposing drugs by exposing every type of cell to every compound in the US Pharmacopoeia to look for novel effects (via transcriptomics)[2] Genomic sequencing, microfluidic liquid handling[4,7], and novel statistical approaches allow for scientific knowledge to be generated at an astounding pace. After the data exist, then we can make hypotheses. This is the new model for biomedical big science. This model of science is often criticized for not being hypothesis-driven, either as "stamp collecting," or as a force for vacuuming up funding that would otherwise support myriad small labs. It does bear fruit for schizophrenia[8], depression [9], as well as diabetes, obesity, and heart disease [10]. Where are the most interesting, most complex, and most challenging data problems these days? Not advertising. Not finance. Not crypto. Biology. (And many from cryptography, math, CS, and physics have switched gears to work on biological problems.) We run on messy stochastic software, and we finally have the tools to read the source code and save edits. It's an exciting time. 1. https://www.broadinstitute.org/about-us https://www.broadinstitute.org/about-us 2. https://clue.io/repurposing https://clue.io/repurposing 3. https://www.gtexportal.org/home/ https://www.gtexportal.org/home/ 4. https://www.broadinstitute.org/news/dronc-seq-microfluidic-single-nucleus-rna-sequencing https://www.broadinstitute.org/news/dronc-seq-microfluidic-s... 5. https://www.broadinstitute.org/news/international-human-cell-atlas-initiative-gets-underway https://www.broadinstitute.org/news/international-human-cell... 6. https://www.broadinstitute.org/klarman-cell-observatory/approach https://www.broadinstitute.org/klarman-cell-observatory/appr... 7. https://www.broadinstitute.org/files/patents/WO2016149661.pdf https://www.broadinstitute.org/files/patents/WO2016149661.pd... 8. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4112379/pdf/emss-59304.pdf https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4112379/pdf/ems... 9. https://www.nature.com/articles/s41588-018-0090-3 https://www.nature.com/articles/s41588-018-0090-3 10. (Via profile of common author in several fields) https://scholar.google.com/citations?hl=en&user=EISUuucAAAAJ&view_op=list_works https://scholar.google.com/citations?hl=en&user=EISUuucAAAAJ...

fizixer 9y ago

> The success of a grant proposal shouldn’t hinge on whether the research is driven by a hypothesis, especially in the physical sciences. You're looking at it the wrong way. If one call for proposal (meaning availability to fund only one project) receives a hundred proposals, then the acceptance rate is going to be 1% no matter what. Requirements like hypothesis inclusion are just ways to split the weed out work into two phases, e.g., eliminate 50 of the 100 proposals because they don't have hypothesis, and then eliminate 49 out of the remaining 50 after more careful reading. If you don't have the hypothesis requirement, that only means 99 out of 100 are eliminated after full reading. And what do you think the likelihood is going to be that the reviewers would accept your hypothesis-free proposal over dozens others that have one (all other things considered equal)? The problem is lack of funding (or alternatively, too many researchers and not enough research work to go around, depending on how you look at it; Masters degree is the new Bachelors degree, PhD is the new Masters, etc, etc). What you're focusing on are marginal issues.

firasd 9y ago

I finally 'got' the concept of hypotheses after a couple stupid public policy experiments here in India. The 'Odd-Even' car rationing experiment in Delhi to reduce pollution was praised because it reduced traffic. Then the country-wide demonetization disaster meant to reduce undeclared income was later touted as a way to enable digital transactions. So to generalize, if you're not clear on what you're trying to achieve, you can claim success by retrofitting your results to any stated purpose.