7 ms·

> A null finding might not be great for the researcher The fix for that would be to change science funding so a null finding doesn't hurt the researcher's care

• dmichulke 8y ago
  It already doesn't, null findings are simply not published. Unless of course you want to reward null findings which might, depending on the circumstances, open another can of worms.
  • thiagotomei 8y ago
    At least in my field (high energy physics in the LHC), null findings are routinely published. Everybody who is searching for physics beyond the standard model publishes their null findings left and right.
    • whatshisface 8y ago
      Above, a commenter makes a distinction between negative and null result, saying that null results are those without any statistically significant conclusion (neither negative nor positive). Those don't get published, the BSM papers usually rule out parts of parameter space which makes them negative results.
• taurath 8y ago
  The interests of humanity and the interests of the organizations that have the capital to fund science projects are not frequently aligned.
  • justinjlynn 8y ago
    This must change.
  • 08-15 8y ago
    Exactly! Stop government funded research! Today, most science is funded by government agencies, like the NIH. This means, bureaucrats who know nothing about science allocate other people's money to research they don't care about. Since they don't understand the research, they need objective measures of "research quality", and they picked number of publications and impact factor of the journals as measures. What you measure becomes your objective, and we got "publish or perish" and all sorts of scientific misconduct. Science needs a free market, too.
    • pilsetnieks 8y ago
      I'm going to hope this was poorly phrased sarcasm.
      • 08-15 8y ago
        Is there anything you disagree with? Or do you simply dislike the conclusion? Five million euro of tax payer money have been blown on a Neanderthal genome. All we got out of it is a hyper sensitive analysis that picked up an artifact (source: first hand experience with the raw data) and called it admixture. But it's a high profile publication thanks to the catchy headline and more tax payer money flows in the same direction. This wouldn't have happened if the bureaucrats who presided over the grant money knew anything about the science and looked into the methods. Or maybe it would have happened anyway, because this was never science but just a publicity stunt. Either way, funding "science" this way is bad.
    • dekhn 8y ago
      Most program managers at NIH are ex-scientists with expertise in the fields they are funding.
• azernik 8y ago
  Well, we do still want to reward researchers for coming up with good hypotheses. It's just that we want to also make it harder to game the metrics.
  • pgeorgi 8y ago
    "I have not failed 700 times. I have not failed once. I have succeeded in proving that those 700 ways will not work. When I have eliminated the ways that will not work, I will find the way that will work." -- Edison By publishing null results you avoid 100 scientists going the same endearing but unproductive routes, which can also help build good hypotheses and speed up progress.
    • 08-15 8y ago
      "Absence of evidence is not evidence of absence." A null result (no significant evidence for anything, a p value above 0.05) is truly null. It doesn't even confirm the null hypothesis. Common wisdom says that Statistical Hypothesis Inference Testing doesn't work because researchers engage in "P Hacking". That's a half truth, researchers really don't understand the meaning of the p-value.
      • wnoise 8y ago
        Which is completely bogus. Absence of evidence, when evidence is expected is indeed (possibly weak) evidence of absence. The stronger "absence of proof is proof of absence" would be fallacious.
      • antidesitter 8y ago
        > Absence of evidence is not evidence of absence. A more accurate way to state this is “Absence of evidence is not always evidence of absence”. Or “Absence of evidence may or may not be evidence of absence depending on the circumstance”, though it doesn’t roll off the tongue as easily.
        • 08-15 8y ago
• syrrim 8y ago
  Never gonna happen. Think about it: the principle of science is mapping the empirical world with a logical one. A null result represents a failure to do this. In a binary case (either this or that), a null result gives us information, but the majority of null results tell us nothing more than "our hypothesis was wrong". Having a wrong hypothesis means you got unlucky sure - but repeated enough times and treated statistically it means you are a less skilled scientist than someone who gets more affirmative results.
  • pbhjpbhj 8y ago
    You're in a maze of twisty passages. If someone can tell you 'turning left is just a wall' then you can focus on the other routes.
    • TeMPOraL 8y ago
      But in reality, it's more like you were blind in a forest, and that someone was operating a LIDAR with binary output (there is/isn't something less than 50 meters ahead). They tell you that at 120.001 degrees there is something. Then they tell you that at 120.002 degrees there is something. Etc. If null results would be treated equally to a positive finding, then people would just spam null results.
      • pbhjpbhj 8y ago
        You're absolutely right. Isn't it good that my rubbish attempt was there to stimulate your useful one ^_^ There's something to be said for brute force on rare occasions though.
  • cabalamat 8y ago
    > Think about it: the principle of science is mapping the empirical world with a logical one. A null result represents a failure to do this. Let's say you try a particular educational intervention. it turns out that if has no effect on children's learning. This is really useful information, because it means that we now know that there is no point in schools trying it.
• drb91 8y ago
  Science should be publicly funded. Capitalist funding of academic research has perverse incentives.
  • apathy 8y ago
    Most of it is.
    • drb91 8y ago
      Yes, I did not mean to imply otherwise, my apologies.
  • dagw 8y ago
    That in itself wouldn't change the incentives too much. Government grant boards don't like that you 'waste' their money either and come up with 'nothing'.
    • drb91 8y ago
      We can elect our officials; we do not choose who to make rich, or what they do with their money, unfortunately.
• 08-15 8y ago
  The fix is to use Bayesian statistics, where every finding updates you beliefs and therefore there is no such thing as a null result. The cult of the p value needs to end.
  • jononor 8y ago
    Are there any practical introductions/guides to applying this method? Interested for my own research.
    • 08-15 8y ago
      E.T.Jaynes: "Probability Theory, The Logic Of Science" A great introduction to the topic and a great general guideline. It's not a collection of recipes, more like a nudge in a direction where you don't need canned recipes anymore.
  • pbhjpbhj 8y ago
    Surely that only works if your hypothesis is partly right?
    • 08-15 8y ago
      I don't know how a hypothesis can be partly right, but I'll read this as "Surely that only works if your prior belief in your hypothesis is neither exactly 0 nor 1?" And that's correct. Prior distributions are never completely concentrated at either extreme, for those could never be updated. If your hypothesis is now totally wrong, your confidence in it (the posterior probability) will converge toward zero.
      • jonnycomputer 8y ago
        If a hypothesis is considered as a species of proposition, then yes, there are a variety of ways in which a hypothesis can be "partly true". Firstly, if the hypothesis is a conjunction, the some of the conjuncts could be true. Propositions can also contain vague predicates with ambiguous truth values. Finally, there is the truthlikeness of the propositions. Some propositions seem to be more true, or have more versimiltude, than others (for which I refer you to https://plato.stanford.edu/entries/truthlikeness/ https://plato.stanford.edu/entries/truthlikeness/)
• BeetleB 8y ago
  >The fix for that would be to change science funding so a null finding doesn't hurt the researcher's career. I get what you're saying, but the counter to that is it would be too easy for scientists to conjure up studies that they have no reason to believe is true, do an experiment, publish the null finding (that frankly, they were expecting), and ask for more money. It's almost trivial to think up experiments that will give you a null finding.
• jonnycomputer 8y ago
  Null results are, almost by definition, easier to find that statistically significant ones. What you need are highly improbably null-results. Null results that are truly surprising, surprising because the preponderance of other evidence, theoretical and realized, point to it being true. In other words, its generally much easier for your head to come out of the barrel without an apple.