3 ms·
Statistician here. Saying nothing about the substance of the argument or whether their data source appropriately classifies what counts as this kind of crime,
by waldrews 14y ago
Statistician here. Saying nothing about the substance of the argument or whether their data source appropriately classifies what counts as this kind of crime, the methodology is on the right track but not optimal.
Of course binning the events by year throws away data about specific timing. Having actual event times would allow fitting a hazard rate model.
There are two simple alternative hypotheses - explanations for a deviation from the Poisson assumption. What they're calling "random" is really "occuring at a homogeneous rate;" so we have to ask - as opposed to what?
1) clustering/overdispersion, because the events happen more often alongside each other (copycat effect or whatever; risk of a new event is a function of time since last event)
2) secular trend (the rate of events is changing over time)
We can't really distinguish between these two without explicitly modeling, and it doesn't look like we have enough data to do that. The tool to do it would be a generalized linear model with an overdispersed Poisson dependent variable.
It's kind of bad form to estimate the Poisson mean from the data, and then use that to fit the distribution you're testing against. You're using the data twice, so the p-value isn't what you think it is.
You should be conditioning on the total number of events in the sample.
Also, chi-square distribution comparison is for large samples. This is a meh-kinda-borderline-midsize sample, to use a technical term.
The test they want that has neither of these problems is Fisher's exact test for equal proportions.
If the events were generated by a Poisson process, the number in each year would be conditionally binomial, and they'd add up to the total observed event count. The test is: is the data explained by all the years getting events randomly at the same rate (null hypothesis) vs. each year having its own rate (alt hypothesis).
And finally, yes, you get a p-value. But you also need to think about the power of the test detect an anomaly if there was one (type II error). For something like this, you could do power analysis by a simple simulation, but you'd need to specify what kind of anomaly you'd want power to detect (e.g. all events cluster together in one year, to pick an extreme).
Otherwise, all you have is a design that has a correct p-value. If you reject the null hypothesis at 5%, but you have low power, you might as well toss a coin and declare the data "nonrandom" 5% of of the time. This might be as good a test as we can get with the data available, but p-value isn't the only thing you can look at.