4 ms·

People often have trouble applying statistical methods correctly, and perhaps often can manipulate statistics to tell a given story. And indeed, P=0.03 on its

People often have trouble applying statistical methods correctly, and perhaps often can manipulate statistics to tell a given story. And indeed, P=0.03 on its own is meaningless without an understanding of how a study is set up and a plausible hypothesis. But inferential statistics is grounded in sound theory, and used correctly and with appropriate assumptions is a powerful tool to reason about data. Without it, how are you supposed to reason about (for example) study results? Appeal to intuition? It seems to me that significance tests are not all-powerful, or foolproof, but are still a very valuable tool.

deleted 12y ago
[deleted]
jwmerrill 12y ago
> Without [inferential statistics], how are you supposed to reason about (for example) study results? Appeal to intuition? The alternative, which the author of the linked article mentions, but doesn't really emphasize, is to report a best estimate of whatever effect you are trying to measure, along with some measure of uncertainty in that estimate. For example, instead of "we failed to find significant evidence that right turn on red increases the expected number of fatalities," you say "our best estimate of the expected increase in fatalities due to right turn on red is 200 +/- 210." This approach puts the most relevant information front and center, and it seems to me, encourages better intuitive reasoning. It's what engineers and most of the hard sciences do most of the time. You do also need to say something about the meaning of your uncertainty estimate (e.g. it's 1 sigma, or 2 sigma, or 95%), or alternatively, there needs to be an understood convention for your field.
- deleted 12y ago
  [deleted]
- vancan1ty 12y ago
  Is the method you described not another form of inferential statistics? (Definitely not hypothesis testing however). It seems that this is a combination of several techniques: 1. First, we either explicitly or implicity choose a model to relate deaths and RTOR laws (perhaps a linear relationship, i.e. [deaths w/ RTOR] = a*[deaths w/o RTOR]). 2. Then we perform point estimation to estimate the parameter "a". 3. Then we compute a confidence interval. With respect to the RTOR example in the paper, it seems to me that it WOULD be incorrect to reject the null hypothesis that the change in crash numbers arises from random chance for ANY INDIVIDUAL STUDY. In this case it seems that you must figure out a way to transfer information between studies to establish this idea of "statistical significance." Perhaps a survey of studies or usage of bayesian techniques would have resolved the difficulty.

deleted 12y ago

[deleted]

jwmerrill 12y ago

> Without [inferential statistics], how are you supposed to reason about (for example) study results? Appeal to intuition? The alternative, which the author of the linked article mentions, but doesn't really emphasize, is to report a best estimate of whatever effect you are trying to measure, along with some measure of uncertainty in that estimate. For example, instead of "we failed to find significant evidence that right turn on red increases the expected number of fatalities," you say "our best estimate of the expected increase in fatalities due to right turn on red is 200 +/- 210." This approach puts the most relevant information front and center, and it seems to me, encourages better intuitive reasoning. It's what engineers and most of the hard sciences do most of the time. You do also need to say something about the meaning of your uncertainty estimate (e.g. it's 1 sigma, or 2 sigma, or 95%), or alternatively, there needs to be an understood convention for your field.

deleted 12y ago
[deleted]
vancan1ty 12y ago
Is the method you described not another form of inferential statistics? (Definitely not hypothesis testing however). It seems that this is a combination of several techniques: 1. First, we either explicitly or implicity choose a model to relate deaths and RTOR laws (perhaps a linear relationship, i.e. [deaths w/ RTOR] = a*[deaths w/o RTOR]). 2. Then we perform point estimation to estimate the parameter "a". 3. Then we compute a confidence interval. With respect to the RTOR example in the paper, it seems to me that it WOULD be incorrect to reject the null hypothesis that the change in crash numbers arises from random chance for ANY INDIVIDUAL STUDY. In this case it seems that you must figure out a way to transfer information between studies to establish this idea of "statistical significance." Perhaps a survey of studies or usage of bayesian techniques would have resolved the difficulty.