4 ms·

So add a per-county term to both fixed and random effects and see if that fits substantially better. This isn't rocket surgery. Simpson's paradox can be straig

by apathy 10y ago

So add a per-county term to both fixed and random effects and see if that fits substantially better. This isn't rocket surgery. Simpson's paradox can be straightforwardly addressed if the stratifying or confounding factor can be identified, and here that is clearly the cases. It appears that the authors did in fact address this.

chris_va 10y ago
If I understand what you saying (apologies if not, it has been a while since I did modelling), that would only give a better fit if the level of segregation/crime was consistent between counties and not already taken into account by their other variables (like inequality). Or the paradox segments were at a county level instead of locally. If be happier if they just got the police records for all 721 cases, and did a more local analysis. Let me know if I'm missing something (I probably am).
- apathy 10y ago
  If you fit a multilevel mixed model, either the intercept or the slopes can vary, but the model fit at each level (and accuracy of predictions on held-out data) will improve substantially if the terms matter and you include them. In this case I would only believe the result if it led to across the board improvement. Contrarily, if a stratified fit's cross-county predictions were dramatically shittier than within county, I might believe it, but by pooling information in a Bayesian framework these authors appear to have aimed at a happy medium. The multilevel aspect is important (country -> county -> locale) because of the shrinkage you get as a side effect of propagating each estimate. If time permits I will look at their code and data (available in the supplement, the author appears to have used Rstan for the fit, although a Python person could probably get a similar result with PyMC or PyStan) and try fiddling some of the dials. The write up is a little wordy, but the model and data is all there for anyone genuinely interested in sensitivity analysis or alterntive fits. They're not hiding anything that I can see.

by apathy 10y ago

chris_va 10y ago

If I understand what you saying (apologies if not, it has been a while since I did modelling), that would only give a better fit if the level of segregation/crime was consistent between counties and not already taken into account by their other variables (like inequality). Or the paradox segments were at a county level instead of locally. If be happier if they just got the police records for all 721 cases, and did a more local analysis. Let me know if I'm missing something (I probably am).

apathy 10y ago
If you fit a multilevel mixed model, either the intercept or the slopes can vary, but the model fit at each level (and accuracy of predictions on held-out data) will improve substantially if the terms matter and you include them. In this case I would only believe the result if it led to across the board improvement. Contrarily, if a stratified fit's cross-county predictions were dramatically shittier than within county, I might believe it, but by pooling information in a Bayesian framework these authors appear to have aimed at a happy medium. The multilevel aspect is important (country -> county -> locale) because of the shrinkage you get as a side effect of propagating each estimate. If time permits I will look at their code and data (available in the supplement, the author appears to have used Rstan for the fit, although a Python person could probably get a similar result with PyMC or PyStan) and try fiddling some of the dials. The write up is a little wordy, but the model and data is all there for anyone genuinely interested in sensitivity analysis or alterntive fits. They're not hiding anything that I can see.