3 ms·

Finally, analysis of police shooting data as a function of county-level predictors suggests that racial bias in police shootings is most likely to emerge in pol

Finally, analysis of police shooting data as a function of county-level predictors suggests that racial bias in police shootings is most likely to emerge in police departments in larger metropolitan counties with low median incomes and a sizable portion of black residents, especially when there is high financial inequality in that county. There is no relationship between county-level racial bias in police shootings and crime rates (even race-specific crime rates), meaning that the racial bias observed in police shootings in this data set is not explainable as a response to local-level crime rates. With the disclaimer that the conclusion might still be correct, I think looking at the county level is completely absurd. You leave yourself open to the Simpson's Paradox at a neighborhood level. For argument's sake, let's say that the majority of police shootings happen in poor neighborhoods. Let's also assume, sadly, that the ratio of black/white people in poor neighborhoods is high. Their analysis would imply that their is a racial bias to the shooting, when in fact, the racial bias could be entirely explained by the demographics. Or it might not, but doing it at a county level completely washes out all useful signal.

apathy 10y ago
So add a per-county term to both fixed and random effects and see if that fits substantially better. This isn't rocket surgery. Simpson's paradox can be straightforwardly addressed if the stratifying or confounding factor can be identified, and here that is clearly the cases. It appears that the authors did in fact address this.
- chris_va 10y ago
  If I understand what you saying (apologies if not, it has been a while since I did modelling), that would only give a better fit if the level of segregation/crime was consistent between counties and not already taken into account by their other variables (like inequality). Or the paradox segments were at a county level instead of locally. If be happier if they just got the police records for all 721 cases, and did a more local analysis. Let me know if I'm missing something (I probably am).
  - apathy 10y ago
    If you fit a multilevel mixed model, either the intercept or the slopes can vary, but the model fit at each level (and accuracy of predictions on held-out data) will improve substantially if the terms matter and you include them. In this case I would only believe the result if it led to across the board improvement. Contrarily, if a stratified fit's cross-county predictions were dramatically shittier than within county, I might believe it, but by pooling information in a Bayesian framework these authors appear to have aimed at a happy medium. The multilevel aspect is important (country -> county -> locale) because of the shrinkage you get as a side effect of propagating each estimate. If time permits I will look at their code and data (available in the supplement, the author appears to have used Rstan for the fit, although a Python person could probably get a similar result with PyMC or PyStan) and try fiddling some of the dials. The write up is a little wordy, but the model and data is all there for anyone genuinely interested in sensitivity analysis or alterntive fits. They're not hiding anything that I can see.

apathy 10y ago

So add a per-county term to both fixed and random effects and see if that fits substantially better. This isn't rocket surgery. Simpson's paradox can be straightforwardly addressed if the stratifying or confounding factor can be identified, and here that is clearly the cases. It appears that the authors did in fact address this.

chris_va 10y ago
If I understand what you saying (apologies if not, it has been a while since I did modelling), that would only give a better fit if the level of segregation/crime was consistent between counties and not already taken into account by their other variables (like inequality). Or the paradox segments were at a county level instead of locally. If be happier if they just got the police records for all 721 cases, and did a more local analysis. Let me know if I'm missing something (I probably am).
- apathy 10y ago
  If you fit a multilevel mixed model, either the intercept or the slopes can vary, but the model fit at each level (and accuracy of predictions on held-out data) will improve substantially if the terms matter and you include them. In this case I would only believe the result if it led to across the board improvement. Contrarily, if a stratified fit's cross-county predictions were dramatically shittier than within county, I might believe it, but by pooling information in a Bayesian framework these authors appear to have aimed at a happy medium. The multilevel aspect is important (country -> county -> locale) because of the shrinkage you get as a side effect of propagating each estimate. If time permits I will look at their code and data (available in the supplement, the author appears to have used Rstan for the fit, although a Python person could probably get a similar result with PyMC or PyStan) and try fiddling some of the dials. The write up is a little wordy, but the model and data is all there for anyone genuinely interested in sensitivity analysis or alterntive fits. They're not hiding anything that I can see.