5 ms·

> forbid the use of any AI/ML in predictive policing I don't get what this means. If you use basic statistical methods and realize that murders are more likely

by bko 5y ago

> forbid the use of any AI/ML in predictive policing I don't get what this means. If you use basic statistical methods and realize that murders are more likely to occur in this location between these hours and adjust your police force accordingly, is this bad? Shouldn't police use basic statistical methods in determining effective policies?

Closi 5y ago
> If you use basic statistical methods and realize that murders are more likely to occur in this location between these hours and adjust your police force accordingly, is this bad? It depends - These models are often non-transparent in terms of how they calculate, so what if the AI/model decides/calculate that a higher percentage of minorities in an area is a high predictor of crime, thus indirectly creates a policy that, for instance, areas with high levels of ethnic minorities require higher levels of policing or harsher restrictions (i.e. curfews, more CCTV e.t.c.). If it's just a basic 'well murders happen between these hours on this street so we will put a police officer there' then of course that should happen, but that's not really policing based on AI/ML, it's just based on basic statistics.
- _ke2e 5y ago
  Where do you draw the line between statistics and ML?
  - brightly-salty 5y ago
    When you can no longer transparently explain the reasoning behind a decision made based on the statistics, and instead have to just shrug and say “because the model told us so”. Accountability is very important in this issue.
    - bko 5y ago
      Apart from neural networks most models are somewhat transparent in terms of reasoning. Neural networks would be the wrong model to use for this anyway. For instance, decision trees can tell us if the population density between this range and the percentage of single family residence is between this range, the probability of a package being stolen is X. Or regression could tell us that the square of the population density is positively correlated to this crime .
      - Closi 5y ago
        This law isn’t talking about banning decision trees or banning regression analysis though, it’s talking about ML being used in specific listed applications.
      - brightly-salty 5y ago
        Exactly. We don’t disagree, I don’t think. Decision trees and regression analysis should count as statistics for the purpose of this issue, as they are transparent and accountable. Neural networks are another beast entirely, in my opinion.
  - Closi 5y ago
    I think they are two different things - statistical modelling to me is about understanding relationships between data (i.e. we are highly confident that X and Y are correlated, and can infer a reason and thus put a policy around that based on open reasoning), and ML is about making predictions in new situations based on historical training data from different situations (but we can't necessarily infer the why).
- bko 5y ago
  > It depends - These models are often non-transparent in terms of how they calculate, so what if the AI/model decides/calculate that a higher percentage of minorities in an area is a high predictor of crime, thus indirectly creates a policy that, for instance, areas with high levels of ethnic minorities require higher levels of policing or harsher restrictions (i.e. curfews, more CCTV e.t.c.). It's pretty simple to make the models transparent. Just select the inputs you want the model to have access to. You shouldn't be feeding your model the percentage of minorities in an area. What's not transparent is how humans make decisions. You can't tell a human to consider only these characteristics but not these others ones. > If it's just a basic 'well murders happen between these hours on this street so we will put a police officer there' then of course that should happen, but that's not really policing based on AI/ML, it's just based on basic statistics. I don't see how you make a meaningful separation between the two. Either you can use predictive models or not. It's hard to carve out "use predictive models but not too predictive". The more likely scenario is just not using any model and distributing police resources uniformly regardless of need.
  - Closi 5y ago
    > It's pretty simple to make the models transparent. Just select the inputs you want the model to have access to. You shouldn't be feeding your model the percentage of minorities in an area. That doesn't make the model transparent, it just makes the input and output transparent, but it's still a black box. It's nice to say that everyone will ensure that the model isn't fed with information which will allow it to make decisions or be biased based on sex / gender / race / income but in reality accidental racial bias has already shown up multiple times in applications where it should be easier to detect than an invisible 'crime prediction' algorithm that the general public won't have access to but public policy will be implemented on the basis of. > I don't see how you make a meaningful separation between the two. Either you can use predictive models or not. It's hard to carve out "use predictive models but not too predictive". The more likely scenario is just not using any model and distributing police resources uniformly regardless of need. I make a meaningful separation because one has clear logic which can be viewed to be unbiased, and the other has a black box calculating it.
    - 5y ago
- throwawaysea 5y ago
  I am still not clear on why that's a problem. If the goal is to reduce or deter crime, and there's an effective way to do that, why does it matter what correlation it has to outside demographic factors? I would argue that not containing crime in a known high-crime area just because of correlation to 'high levels of ethnic minorities' is itself racist, since that is itself an act of making race a factor in decision making. It also doesn't fit what minorities actually want. As an example, an overwhelming majority of Black Americans want policing to stay the same or increase (https://news.gallup.com/poll/316571/black-americans-police-retain-local-presence.aspx https://news.gallup.com/poll/316571/black-americans-police-r...), rather than decrease, despite what (predominantly-white) activists have been pushing for in the last year.
tsegers 5y ago
Using basic statistical methods to determine effective policies can result in policies that a lot of people would consider unfair and unethical.
- bko 5y ago
  Can you elaborate? For instance, I lived in an area where there was a lot of bike thefts. Everyone knew it and police got a lot of bike theft reports (it was in the statistics). I would like to see more police enforcement to deter bike thefts. I think its unethical to ignore the needs of a neighborhood when allocating police resources.
  - randomswede 5y ago
    But if you then over-intensify the bike theft enforcement in that area, you will continue to have an over-reporting of bike thefts in that area, leading to more intense bike theft enforcement, eventually leading to an unjust and unethical police presence across the municipality.
clairity 5y ago
> “…murders are more likely to occur in this location between these hours…” you’re on the wrong track if murders (and the like) are the crimes you’re targeting. murders, while horrific, are a tiny and very localized danger. the crimes we need to be policing more actively happen around money and power (mass surveillance, corruption, etc.), which are far too common and have broad, highly externalized, and surreptitious effects. i’d heartily support (heavy-handedly, even) applying any and all such techniques on (so-called) white-collar crimes. in contrast, social norming in communities (like not murdering) should be convivial rather than adversarial, as that’s much more effective (as evidenced by how little murder actually happens).
- adolph 5y ago
  > murders, while horrific, are a tiny and very localized danger Because they are naturally uncommon and localized or because they are policed heavily?
  - clairity 5y ago
    the amount of policing has basically no effect. numerous studies show that potential punishments don't deter murder, either pre-meditated or murders of opportunity. further, police rarely ever stop murders, and only solve about 55% of murder cases. lastly, murders are rare: 0.005%, or 16k/yr, in the US. that's because of the severe social, not legal, repercussions. people feeling connected to each other lowers the murder rate, not policing.
    - adolph 5y ago
      I wonder what would happen if murder stopped being policed. Would the murder rate remain unchanged as “numerous studies” suggest? Or would an unchanged base rate in murder without official retribution lead to blood feud style relationships between people and increase a secondary, tertiary, … n-terary murder rates? If police leave 45% of murders unaddressed, I wonder how many current murders are secondary in nature, as in would not have occurred without an unretributed precipitating crime be-it murder, rape, assault, etc. It’s an interesting idea to think about murder not being policed or have legal repercussions. Would it be comforting to know if you kill somebody drunk driving there wouldn’t be a big problem? I guess somebody like Dallas’ Dr. Death would still have to deal with pesky civil lawsuits. https://en.wikipedia.org/wiki/Christopher_Duntsch https://en.wikipedia.org/wiki/Christopher_Duntsch
adolph 5y ago
> Shouldn't police use basic statistical methods in determining effective policies? No. For one, police are not statisticians. Here's a military example from WWII: During World War II, the statistician Abraham Wald took survivorship bias into his calculations when considering how to minimize bomber losses to enemy fire. The Statistical Research Group (SRG) at Columbia University, which Wald was a part of, examined the damage done to aircraft that had returned from missions and recommended adding armor to the areas that showed the least damage. This contradicted the US military's conclusion that the most-hit areas of the plane needed additional armor. Wald noted that the military only considered the aircraft that had survived their missions – ignoring any bombers that had been shot down or otherwise lost, and thus also been rendered unavailable for assessment. The bullet holes in the returning aircraft represented areas where a bomber could take damage and still fly well enough to return safely to base. Therefore, Wald proposed that the Navy reinforce areas where the returning aircraft were unscathed, inferring that planes hit in those areas were the ones most likely to be lost. His work is considered seminal in the then-nascent discipline of operational research. https://en.wikipedia.org/wiki/Survivorship_bias https://en.wikipedia.org/wiki/Survivorship_bias

by bko 5y ago

Closi 5y ago

> If you use basic statistical methods and realize that murders are more likely to occur in this location between these hours and adjust your police force accordingly, is this bad? It depends - These models are often non-transparent in terms of how they calculate, so what if the AI/model decides/calculate that a higher percentage of minorities in an area is a high predictor of crime, thus indirectly creates a policy that, for instance, areas with high levels of ethnic minorities require higher levels of policing or harsher restrictions (i.e. curfews, more CCTV e.t.c.). If it's just a basic 'well murders happen between these hours on this street so we will put a police officer there' then of course that should happen, but that's not really policing based on AI/ML, it's just based on basic statistics.

_ke2e 5y ago
Where do you draw the line between statistics and ML?
- brightly-salty 5y ago
  When you can no longer transparently explain the reasoning behind a decision made based on the statistics, and instead have to just shrug and say “because the model told us so”. Accountability is very important in this issue.
  - bko 5y ago
    Apart from neural networks most models are somewhat transparent in terms of reasoning. Neural networks would be the wrong model to use for this anyway. For instance, decision trees can tell us if the population density between this range and the percentage of single family residence is between this range, the probability of a package being stolen is X. Or regression could tell us that the square of the population density is positively correlated to this crime .
    - Closi 5y ago
      This law isn’t talking about banning decision trees or banning regression analysis though, it’s talking about ML being used in specific listed applications.
    - brightly-salty 5y ago
      Exactly. We don’t disagree, I don’t think. Decision trees and regression analysis should count as statistics for the purpose of this issue, as they are transparent and accountable. Neural networks are another beast entirely, in my opinion.
- Closi 5y ago
  I think they are two different things - statistical modelling to me is about understanding relationships between data (i.e. we are highly confident that X and Y are correlated, and can infer a reason and thus put a policy around that based on open reasoning), and ML is about making predictions in new situations based on historical training data from different situations (but we can't necessarily infer the why).
bko 5y ago
> It depends - These models are often non-transparent in terms of how they calculate, so what if the AI/model decides/calculate that a higher percentage of minorities in an area is a high predictor of crime, thus indirectly creates a policy that, for instance, areas with high levels of ethnic minorities require higher levels of policing or harsher restrictions (i.e. curfews, more CCTV e.t.c.). It's pretty simple to make the models transparent. Just select the inputs you want the model to have access to. You shouldn't be feeding your model the percentage of minorities in an area. What's not transparent is how humans make decisions. You can't tell a human to consider only these characteristics but not these others ones. > If it's just a basic 'well murders happen between these hours on this street so we will put a police officer there' then of course that should happen, but that's not really policing based on AI/ML, it's just based on basic statistics. I don't see how you make a meaningful separation between the two. Either you can use predictive models or not. It's hard to carve out "use predictive models but not too predictive". The more likely scenario is just not using any model and distributing police resources uniformly regardless of need.
- Closi 5y ago
  > It's pretty simple to make the models transparent. Just select the inputs you want the model to have access to. You shouldn't be feeding your model the percentage of minorities in an area. That doesn't make the model transparent, it just makes the input and output transparent, but it's still a black box. It's nice to say that everyone will ensure that the model isn't fed with information which will allow it to make decisions or be biased based on sex / gender / race / income but in reality accidental racial bias has already shown up multiple times in applications where it should be easier to detect than an invisible 'crime prediction' algorithm that the general public won't have access to but public policy will be implemented on the basis of. > I don't see how you make a meaningful separation between the two. Either you can use predictive models or not. It's hard to carve out "use predictive models but not too predictive". The more likely scenario is just not using any model and distributing police resources uniformly regardless of need. I make a meaningful separation because one has clear logic which can be viewed to be unbiased, and the other has a black box calculating it.
  - 5y ago
throwawaysea 5y ago
I am still not clear on why that's a problem. If the goal is to reduce or deter crime, and there's an effective way to do that, why does it matter what correlation it has to outside demographic factors? I would argue that not containing crime in a known high-crime area just because of correlation to 'high levels of ethnic minorities' is itself racist, since that is itself an act of making race a factor in decision making. It also doesn't fit what minorities actually want. As an example, an overwhelming majority of Black Americans want policing to stay the same or increase (https://news.gallup.com/poll/316571/black-americans-police-retain-local-presence.aspx https://news.gallup.com/poll/316571/black-americans-police-r...), rather than decrease, despite what (predominantly-white) activists have been pushing for in the last year.

tsegers 5y ago

Using basic statistical methods to determine effective policies can result in policies that a lot of people would consider unfair and unethical.

bko 5y ago
Can you elaborate? For instance, I lived in an area where there was a lot of bike thefts. Everyone knew it and police got a lot of bike theft reports (it was in the statistics). I would like to see more police enforcement to deter bike thefts. I think its unethical to ignore the needs of a neighborhood when allocating police resources.
- randomswede 5y ago
  But if you then over-intensify the bike theft enforcement in that area, you will continue to have an over-reporting of bike thefts in that area, leading to more intense bike theft enforcement, eventually leading to an unjust and unethical police presence across the municipality.

clairity 5y ago

> “…murders are more likely to occur in this location between these hours…” you’re on the wrong track if murders (and the like) are the crimes you’re targeting. murders, while horrific, are a tiny and very localized danger. the crimes we need to be policing more actively happen around money and power (mass surveillance, corruption, etc.), which are far too common and have broad, highly externalized, and surreptitious effects. i’d heartily support (heavy-handedly, even) applying any and all such techniques on (so-called) white-collar crimes. in contrast, social norming in communities (like not murdering) should be convivial rather than adversarial, as that’s much more effective (as evidenced by how little murder actually happens).

adolph 5y ago
> murders, while horrific, are a tiny and very localized danger Because they are naturally uncommon and localized or because they are policed heavily?
- clairity 5y ago
  the amount of policing has basically no effect. numerous studies show that potential punishments don't deter murder, either pre-meditated or murders of opportunity. further, police rarely ever stop murders, and only solve about 55% of murder cases. lastly, murders are rare: 0.005%, or 16k/yr, in the US. that's because of the severe social, not legal, repercussions. people feeling connected to each other lowers the murder rate, not policing.
  - adolph 5y ago
    I wonder what would happen if murder stopped being policed. Would the murder rate remain unchanged as “numerous studies” suggest? Or would an unchanged base rate in murder without official retribution lead to blood feud style relationships between people and increase a secondary, tertiary, … n-terary murder rates? If police leave 45% of murders unaddressed, I wonder how many current murders are secondary in nature, as in would not have occurred without an unretributed precipitating crime be-it murder, rape, assault, etc. It’s an interesting idea to think about murder not being policed or have legal repercussions. Would it be comforting to know if you kill somebody drunk driving there wouldn’t be a big problem? I guess somebody like Dallas’ Dr. Death would still have to deal with pesky civil lawsuits. https://en.wikipedia.org/wiki/Christopher_Duntsch https://en.wikipedia.org/wiki/Christopher_Duntsch

adolph 5y ago

> Shouldn't police use basic statistical methods in determining effective policies? No. For one, police are not statisticians. Here's a military example from WWII: During World War II, the statistician Abraham Wald took survivorship bias into his calculations when considering how to minimize bomber losses to enemy fire. The Statistical Research Group (SRG) at Columbia University, which Wald was a part of, examined the damage done to aircraft that had returned from missions and recommended adding armor to the areas that showed the least damage. This contradicted the US military's conclusion that the most-hit areas of the plane needed additional armor. Wald noted that the military only considered the aircraft that had survived their missions – ignoring any bombers that had been shot down or otherwise lost, and thus also been rendered unavailable for assessment. The bullet holes in the returning aircraft represented areas where a bomber could take damage and still fly well enough to return safely to base. Therefore, Wald proposed that the Navy reinforce areas where the returning aircraft were unscathed, inferring that planes hit in those areas were the ones most likely to be lost. His work is considered seminal in the then-nascent discipline of operational research. https://en.wikipedia.org/wiki/Survivorship_bias https://en.wikipedia.org/wiki/Survivorship_bias