3 ms·
Null findings are often not as useful as you might think. A point null hypothesis for a continuous variable is literally always false. Especially for the softe
by SquishyPanda23 8y ago
Null findings are often not as useful as you might think.
A point null hypothesis for a continuous variable is literally always false. Especially for the softer science, I've even seen studies mocked for having too many data points, since it's known that with enough data null hypotheses are false.
The story is better if your null hypothesis is an interval, but then you're really just obliquely using the interval to bound something you could be measuring more directly anyway.
What I'd like to see is moving away from null hypothesis testing altogether and focusing on measuring things. For example, focusing on measuring effect sizes, or the probability that a hypothesis is true.
- thiagotomei 8y agoBut what does "measuring more directly anyway" mean if you're trying to measure something that may or may not exist? For instance, in searches for new phenomena in high-energy physics, one usually puts an upper limit on deviations from the expectation of "known physics" (i.e., standard model). That essentially translates to statements like "if this particle exists, its mass should be higher than X TeV, or else we would have seen it already in our data". Of course, in reality, the particle probably does not exist, so you cannot really measure its mass!
- SquishyPanda23 8y agoSorry this is so late, but you can measure the probability that the particle exists. Null hypothesis tests basically try to calculate the probability of a data set given the null hypothesis. What you really want is the probability of a hypothesis given the data set. So in that case, you want to estimate the probability of theories of physics, such as those that include the particle and those that don't.
- wnoise 8y agos/probability/odds ratio/