4 ms·

I'd go a step back from "flawed data". I'd call it a bad sample size. See [Researchers have long known that whites are more likely to use and sell drugs. And y

by Forge36 10y ago

I'd go a step back from "flawed data". I'd call it a bad sample size. See [Researchers have long known that whites are more likely to use and sell drugs. And yet, who is arrested for drugs? Blacks. 13% of the US population is black, but over 60% of those in prison are black.] So how would this happen? One possibility: if a crime happens more police frequent there. So you find more crime. Because we've over-sampled black neighborhoods (including DNA testing making it more likely to find relatives of someone already in prison) the mistaken belief becomes confirmed, and reinforced by the continued arrests. I'm unaware of how to call out bad sample sizes in data. However doing so may help highlight invalid assumptions.

danielvf 10y ago
You said "13% of the US population is black, but over 60% of those in prison are black." Blacks have a six times greater homicide rate than the rest of the US population, according to FBI extended homicide data. Almost half of homicides in the US in the last year I checked were by African Americans. With the long prison sentences given for violent crimes, violent crimes are disproportionately represented in prison populations. As a thought experiment, imagine that each year one person with a thirty year sentance was committed, and one person with a one year sentence. One the system stabled out, 97% of the prison population would be serving 30 year terms.
- Forge36 10y ago
  That's also very good to point out. Please note I quoted the article. You point out a bias in the statistic. Of the imprisoned population. What are the percentage breakdowns for types of crime?
  - danielvf 10y ago
    I did some research on this last year. Here's what I found: https://news.ycombinator.com/item?id=10318406 https://news.ycombinator.com/item?id=10318406
  - Nadya 10y ago
    You can see 4 categories here: http://felonvoting.procon.org/view.resource.php?resourceID=004339 http://felonvoting.procon.org/view.resource.php?resourceID=0... Drugs and violent crime are the majority. So now we can get a breakdown of race. You can see a minor (and slightly misleading, I'll explain) breakdown by race here: https://www.fbi.gov/about-us/cjis/ucr/crime-in-the-u.s/2011/crime-in-the-u.s.-2011/tables/table-43 https://www.fbi.gov/about-us/cjis/ucr/crime-in-the-u.s/2011/... The reason the above table is "a bit misleading" is because FBI tracks "Hispanic" as "White". Most people tend to think of "Hispanic" being separate. You'll need to investigate a bit for accurate data/studies regarding this (because I'm not going to while at work), but if I recall "Hispanic" commit roughly equivalent amount of crime as "Black". Note that this is arrests, not convictions, and it is also possible there is racial bias in courts where Hispanic/Blacks are more likely to be convicted for violent/drug-related crimes and thus over-represent themselves in prisons. Honestly, the data isn't useful because there are too many unknown variables to draw any meaningful conclusions but many people with racially motivated beliefs love to point at this data.
- Joof 10y ago
  Homicide is a small percentage of violent crimes (close to 0.1%), so it isn't necessarily what skews the prison population. Thankfully, by your logic, violent crime in general would do that, but it becomes much less clear using that data. The gap in violent crime is much smaller than 6 times according to this DOJ report: http://www.bjs.gov/content/pub/pdf/cv14.pdf http://www.bjs.gov/content/pub/pdf/cv14.pdf summed up in: http://www.amren.com/news/2015/07/new-doj-statistics-on-race-and-violent-crime/ http://www.amren.com/news/2015/07/new-doj-statistics-on-race... Comparing the violent crime rates to the % population in the US (63.7% white pop; 42.9% offenders | 12.6% black pop; 22.4% offenders), it's roughly 2.8 times as high for black as whites per person. Bad, but not quite 6 times as high by any measure. Going back to the original number of total offenders, it would make sense (assuming violent crime is represented disproportionately), that roughly 42% of those imprisoned are white and 22.4% are black; instead blacks account for a whopping 37% of the prison population and whites only 22%. It would seem something more is going on here. Honestly I'm not surprised blacks dislike officers either; unarmed blacks are 3.49 time more likely to be shot by an officer according to a bayesian analysis[1]. They also found no relationship between this statistic and crime rate in the area. [1] http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0141854 http://journals.plos.org/plosone/article?id=10.1371/journal....
  - danielvf 10y ago
    Big props to you for digging into the raw stats. To get estimates for prision population percentages you'll need to adjust the convictions for each category by the average number of years served for that crime category. As you said, homicides make up a tiny fraction of crimes and yet a double digit percentage of state prisoners are serving time for homicide.
    - Joof 10y ago
      Fair enough; homicides are indeed ~12% of the state prison population. I can't do it now, but I'm still very curious. There are dozens of other factors that could go into it as well; if one race has a higher average sentencing time or serving time; that would bring up more questions.
aoki 10y ago
i don't think you meant "bad sample size," i think you meant "non-representative sample" or just "bad sampling design." the parent referred to bias. the example you're giving is one where the sample is biased because of a poor choice of sampling method. so, i think you're actually agreeing with the parent (as well as danah).

by Forge36 10y ago

danielvf 10y ago

You said "13% of the US population is black, but over 60% of those in prison are black." Blacks have a six times greater homicide rate than the rest of the US population, according to FBI extended homicide data. Almost half of homicides in the US in the last year I checked were by African Americans. With the long prison sentences given for violent crimes, violent crimes are disproportionately represented in prison populations. As a thought experiment, imagine that each year one person with a thirty year sentance was committed, and one person with a one year sentence. One the system stabled out, 97% of the prison population would be serving 30 year terms.

Forge36 10y ago
That's also very good to point out. Please note I quoted the article. You point out a bias in the statistic. Of the imprisoned population. What are the percentage breakdowns for types of crime?
- danielvf 10y ago
  I did some research on this last year. Here's what I found: https://news.ycombinator.com/item?id=10318406 https://news.ycombinator.com/item?id=10318406
- Nadya 10y ago
  You can see 4 categories here: http://felonvoting.procon.org/view.resource.php?resourceID=004339 http://felonvoting.procon.org/view.resource.php?resourceID=0... Drugs and violent crime are the majority. So now we can get a breakdown of race. You can see a minor (and slightly misleading, I'll explain) breakdown by race here: https://www.fbi.gov/about-us/cjis/ucr/crime-in-the-u.s/2011/crime-in-the-u.s.-2011/tables/table-43 https://www.fbi.gov/about-us/cjis/ucr/crime-in-the-u.s/2011/... The reason the above table is "a bit misleading" is because FBI tracks "Hispanic" as "White". Most people tend to think of "Hispanic" being separate. You'll need to investigate a bit for accurate data/studies regarding this (because I'm not going to while at work), but if I recall "Hispanic" commit roughly equivalent amount of crime as "Black". Note that this is arrests, not convictions, and it is also possible there is racial bias in courts where Hispanic/Blacks are more likely to be convicted for violent/drug-related crimes and thus over-represent themselves in prisons. Honestly, the data isn't useful because there are too many unknown variables to draw any meaningful conclusions but many people with racially motivated beliefs love to point at this data.
Joof 10y ago
Homicide is a small percentage of violent crimes (close to 0.1%), so it isn't necessarily what skews the prison population. Thankfully, by your logic, violent crime in general would do that, but it becomes much less clear using that data. The gap in violent crime is much smaller than 6 times according to this DOJ report: http://www.bjs.gov/content/pub/pdf/cv14.pdf http://www.bjs.gov/content/pub/pdf/cv14.pdf summed up in: http://www.amren.com/news/2015/07/new-doj-statistics-on-race-and-violent-crime/ http://www.amren.com/news/2015/07/new-doj-statistics-on-race... Comparing the violent crime rates to the % population in the US (63.7% white pop; 42.9% offenders | 12.6% black pop; 22.4% offenders), it's roughly 2.8 times as high for black as whites per person. Bad, but not quite 6 times as high by any measure. Going back to the original number of total offenders, it would make sense (assuming violent crime is represented disproportionately), that roughly 42% of those imprisoned are white and 22.4% are black; instead blacks account for a whopping 37% of the prison population and whites only 22%. It would seem something more is going on here. Honestly I'm not surprised blacks dislike officers either; unarmed blacks are 3.49 time more likely to be shot by an officer according to a bayesian analysis[1]. They also found no relationship between this statistic and crime rate in the area. [1] http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0141854 http://journals.plos.org/plosone/article?id=10.1371/journal....
- danielvf 10y ago
  Big props to you for digging into the raw stats. To get estimates for prision population percentages you'll need to adjust the convictions for each category by the average number of years served for that crime category. As you said, homicides make up a tiny fraction of crimes and yet a double digit percentage of state prisoners are serving time for homicide.
  - Joof 10y ago
    Fair enough; homicides are indeed ~12% of the state prison population. I can't do it now, but I'm still very curious. There are dozens of other factors that could go into it as well; if one race has a higher average sentencing time or serving time; that would bring up more questions.

aoki 10y ago

i don't think you meant "bad sample size," i think you meant "non-representative sample" or just "bad sampling design." the parent referred to bias. the example you're giving is one where the sample is biased because of a poor choice of sampling method. so, i think you're actually agreeing with the parent (as well as danah).