3 ms·

>"Furthermore, the neutral theory is not merely a neutral model, to be used as a null hypothesis against which more interesting hypotheses can be tested." What

by nonbel 8y ago

>"Furthermore, the neutral theory is not merely a neutral model, to be used as a null hypothesis against which more interesting hypotheses can be tested." What possible scientific purpose would this serve? Why not just test the interesting hypothesis and leave out the strawman?

ssivark 8y ago
When the model predictions are deterministic and easy to observe, the null hypothesis is not needed. When model predictions are statistical, then, you want to answer the question: how "likely" is this model, given the data? Answering that with a probability (as would be done in a Bayesian approach) is hard because there is no natural "space of models" over which to compute that probability. So, in the frequentist approach, you check whether the data is sufficiently more likely to have resulted from your model, rather than the null hypothesis model. The "sufficiently more likely" is quantified by a p-value.
- nonbel 8y ago
  >"When model predictions are statistical, then, you want to answer the question: how "likely" is this model, given the data?" Huh? Here is a model with statistical predictions. Its a model of flipping a fair coin 20 times a day for 365 days: p = 0.5 nDays = 365 nPerDay = 20 dat = rbinom(nDays, nPerDay, p) hist(dat) https://image.ibb.co/bwuYCT/flips.png https://image.ibb.co/bwuYCT/flips.png Here is how I compare theoretical to actual results: > mean(dat) [1] 10.06575 > var(dat) [1] 5.479181 > > nPerDay*p [1] 10 > nPerDay*p*(1-p) [1] 5 >"So, in the frequentist approach, you check whether the data is sufficiently more likely to have resulted from your model, rather than the null hypothesis model. The "sufficiently more likely" is quantified by a p-value." A p-value says something about the fit of the model used to calculate it. It doesn't say anything about any other models.
  - ssivark 8y ago
    Your analysis tells nothing about whether the model of fair coin being flipped is confirmed. You see a deviation of 0.06 in the mean and 0.47 in the variance. Is that negligible? Imagine you had much less data i.e. nDays=5 — What if your sample had a mean of 10.6 and a variance of 4.8? Could you confirm or rule out the fair coin model? Or confirm/ rule out the 0.6-heads-0.4-tails model? Within the frequenting framework, you’d need a null hypothesis to answer that question. Regarding p-value, I agree I could have phrased my comment better. In the frequentist framework, there is no way to confirm a model. The only way to arrive at a model is to falsify all “simpler” models one can think of, that might possibly have explained the data.
    - nonbel 8y ago
      >"Your analysis tells nothing about whether the model of fair coin being flipped is confirmed. You see a deviation of 0.06 in the mean and 0.47 in the variance. Is that negligible?" Th easiest way is to just run a simulation thousands or millions of times then check how often the observed results occurred. Eg, using R again to get distributions for the mean and sd for the model and getting the one sided p-value: p = 0.5 nDays = 365 nPerDay = 20 obs = rbinom(nDays, nPerDay, p) sim = replicate(1e3, rbinom(nDays, nPerDay, p)) stats = apply(dat, 2, function(x) c(mean(x), var(x))) mean(stats[1,] > mean(obs)) mean(stats[2,] > var(obs)) >"In the frequentist framework, there is no way to confirm a model. The only way to arrive at a model is to falsify all “simpler” models one can think of, that might possibly have explained the data." That isn't frequestist. Its the so-called "NHST hybrid" which it seems confused stats 101 authors in the 1930-1940s cobbled together from the ideas of Ronald Fisher, Jerzy Neyman, and Egon Pearson. This is a decent intro to that topic, but there are thousands of papers on this: http://library.mpib-berlin.mpg.de/ft/gg/GG_Mindless_2004.pdf http://library.mpib-berlin.mpg.de/ft/gg/GG_Mindless_2004.pdf
      - ssivark 8y ago
        Thanks for the Gigerenzer article. While he points out unfortunate problems with the way NHST is commonly/mindlessly used, none of that changes the fact that in many cases, there is a natural null hypothesis from an application of Occam's razor. (which was the point of the response to OP's question)
dahart 8y ago
Are you asking whether null hypotheses have a scientific purpose? A null hypothesis is not a strawman, it is a way to test if your hypothesis might be false, as opposed to testing if it might be true. Null hypotheses in general are considered pretty useful for scientific work, I'd say... they're the cornerstone of falsifiability, without which, you don't actually have real science.
- nonbel 8y ago
  I'm not saying get rid of the null hypothesis. I'm saying set the "interesting hypothesis" as the hypothesis to be "nullified" (ie, tested).
  - dahart 8y ago
    The interesting hypothesis can't be the null hypothesis; that's the point of a null hypothesis.
    - nonbel 8y ago
      Why not? I mean I could go find examples of Ronald Fisher (guy who popularized them) doing this if you want...
      - dahart 8y ago
        You seem to know plenty about it to me, along with the history... probably more than I do, so perhaps I’m not helping. Given that, your original question feels odd. Thinking about what kind of science one might do around evolution, the neutral theory seems like a very natural thing to test against. It seems useful to disprove that a hypothesis might be part of a separate or competing theory. If you wanted to test that something is caused by natural selection, it would be very handy to show that it can’t be random mutation or genetic drift. Anyway, you’re right, the null and alternative hypotheses can sometimes be the same, or to be pedantic, the null has to be at least the logical not of the alternative. But in practice, the alternative is not always nullable, and the null is usually answering a different question. If you know about Fisher, then you also know his approach isn’t all that popular today, and that there’s a long history of people arguing about the benefits of a confidence test that is separate from the alternative hypothesis, the thing you’re trying to prove.

by nonbel 8y ago

ssivark 8y ago

When the model predictions are deterministic and easy to observe, the null hypothesis is not needed. When model predictions are statistical, then, you want to answer the question: how "likely" is this model, given the data? Answering that with a probability (as would be done in a Bayesian approach) is hard because there is no natural "space of models" over which to compute that probability. So, in the frequentist approach, you check whether the data is sufficiently more likely to have resulted from your model, rather than the null hypothesis model. The "sufficiently more likely" is quantified by a p-value.

nonbel 8y ago
>"When model predictions are statistical, then, you want to answer the question: how "likely" is this model, given the data?" Huh? Here is a model with statistical predictions. Its a model of flipping a fair coin 20 times a day for 365 days: p = 0.5 nDays = 365 nPerDay = 20 dat = rbinom(nDays, nPerDay, p) hist(dat) https://image.ibb.co/bwuYCT/flips.png https://image.ibb.co/bwuYCT/flips.png Here is how I compare theoretical to actual results: > mean(dat) [1] 10.06575 > var(dat) [1] 5.479181 > > nPerDay*p [1] 10 > nPerDay*p*(1-p) [1] 5 >"So, in the frequentist approach, you check whether the data is sufficiently more likely to have resulted from your model, rather than the null hypothesis model. The "sufficiently more likely" is quantified by a p-value." A p-value says something about the fit of the model used to calculate it. It doesn't say anything about any other models.
- ssivark 8y ago
  Your analysis tells nothing about whether the model of fair coin being flipped is confirmed. You see a deviation of 0.06 in the mean and 0.47 in the variance. Is that negligible? Imagine you had much less data i.e. nDays=5 — What if your sample had a mean of 10.6 and a variance of 4.8? Could you confirm or rule out the fair coin model? Or confirm/ rule out the 0.6-heads-0.4-tails model? Within the frequenting framework, you’d need a null hypothesis to answer that question. Regarding p-value, I agree I could have phrased my comment better. In the frequentist framework, there is no way to confirm a model. The only way to arrive at a model is to falsify all “simpler” models one can think of, that might possibly have explained the data.
  - nonbel 8y ago
    >"Your analysis tells nothing about whether the model of fair coin being flipped is confirmed. You see a deviation of 0.06 in the mean and 0.47 in the variance. Is that negligible?" Th easiest way is to just run a simulation thousands or millions of times then check how often the observed results occurred. Eg, using R again to get distributions for the mean and sd for the model and getting the one sided p-value: p = 0.5 nDays = 365 nPerDay = 20 obs = rbinom(nDays, nPerDay, p) sim = replicate(1e3, rbinom(nDays, nPerDay, p)) stats = apply(dat, 2, function(x) c(mean(x), var(x))) mean(stats[1,] > mean(obs)) mean(stats[2,] > var(obs)) >"In the frequentist framework, there is no way to confirm a model. The only way to arrive at a model is to falsify all “simpler” models one can think of, that might possibly have explained the data." That isn't frequestist. Its the so-called "NHST hybrid" which it seems confused stats 101 authors in the 1930-1940s cobbled together from the ideas of Ronald Fisher, Jerzy Neyman, and Egon Pearson. This is a decent intro to that topic, but there are thousands of papers on this: http://library.mpib-berlin.mpg.de/ft/gg/GG_Mindless_2004.pdf http://library.mpib-berlin.mpg.de/ft/gg/GG_Mindless_2004.pdf
    - ssivark 8y ago
      Thanks for the Gigerenzer article. While he points out unfortunate problems with the way NHST is commonly/mindlessly used, none of that changes the fact that in many cases, there is a natural null hypothesis from an application of Occam's razor. (which was the point of the response to OP's question)

dahart 8y ago

Are you asking whether null hypotheses have a scientific purpose? A null hypothesis is not a strawman, it is a way to test if your hypothesis might be false, as opposed to testing if it might be true. Null hypotheses in general are considered pretty useful for scientific work, I'd say... they're the cornerstone of falsifiability, without which, you don't actually have real science.

nonbel 8y ago
I'm not saying get rid of the null hypothesis. I'm saying set the "interesting hypothesis" as the hypothesis to be "nullified" (ie, tested).
- dahart 8y ago
  The interesting hypothesis can't be the null hypothesis; that's the point of a null hypothesis.
  - nonbel 8y ago
    Why not? I mean I could go find examples of Ronald Fisher (guy who popularized them) doing this if you want...
    - dahart 8y ago
      You seem to know plenty about it to me, along with the history... probably more than I do, so perhaps I’m not helping. Given that, your original question feels odd. Thinking about what kind of science one might do around evolution, the neutral theory seems like a very natural thing to test against. It seems useful to disprove that a hypothesis might be part of a separate or competing theory. If you wanted to test that something is caused by natural selection, it would be very handy to show that it can’t be random mutation or genetic drift. Anyway, you’re right, the null and alternative hypotheses can sometimes be the same, or to be pedantic, the null has to be at least the logical not of the alternative. But in practice, the alternative is not always nullable, and the null is usually answering a different question. If you know about Fisher, then you also know his approach isn’t all that popular today, and that there’s a long history of people arguing about the benefits of a confidence test that is separate from the alternative hypothesis, the thing you’re trying to prove.