6 ms·

> [algorithms] decide who gets access to credit and who pays higher insurance premiums, as well as who will receive online advertising for luxury handbags versu

by bko 10y ago

> [algorithms] decide who gets access to credit and who pays higher insurance premiums, as well as who will receive online advertising for luxury handbags versus who’ll be targeted by predatory ads for for-profit universities. As opposed to what? Subjective decisions made by humans based on biases and personal preferences with no semblance of reason or fairness? > for example, who lives in an area targeted by crime fighting algorithms that add more police to his neighborhood because of higher violent crime rates will necessarily be more likely to be targeted for any petty violation, which adds to a digital profile that could subsequently limit his credit, his job prospects, and so on. So having more cops in a high-crime neighborhood is somehow a bad thing, because the presence of more police actually confirms the fact that there is more of a need of police in this neighborhood? > this technology was actually siloing people into online gated communities where they no longer had to even acknowledge the existence of the poor, So targeted advertising to the needs of people is bad because... I can't even make sense of this one.

guelo 10y ago
> the presence of more police actually confirms the fact that there is more of a need of police in this neighborhood? If we were to enforce all laws we would need massively more police in every neighborhood. When police are sent to flood a poor neighborhood they end up ticketing poor residents for a bunch of "violations" that they would also find if they went to the rich neighborhoods. But if they ever actually did issue all those ticky-tack tickets in the rich neighborhoods residents would be up in arms contacting their city council members and it would end quickly. In the poor neighborhoods nobody ever hears about it and the residents that can least afford it take the financial hits.
- fufubob 10y ago
  And how would you prove that hypothesis? With some data and an algorithm. The problem you speak of can be fixed by improving the model, not abandoning it.
- burgreblast 10y ago
  > they end up ticketing poor residents for a bunch of > "violations" that they would also find if they went > to the rich neighborhoods. I'm not sure what data you have to support this. But check out violent crime rates. Larceny-thefts. Burglary/Property theft. Motor Vehicle Theft. Aggrevated assult. Robbery (mugging, stickups. etc). These very popular crimes aren't so popular in nicer neighborhoods. Rich neighborhoods have their own problems, but let's not appeal to the spirit of lawlessness (cops are bad, rules are bad, unfairly arrested, etc.)
- IncRnd 10y ago
  There have been zero shootings in the area where I live. More police patrols will not increase the number of shootings. It makes sense for police to be where serious crime occurs.
hood_syntax 10y ago
> So having more cops in a high-crime neighborhood is somehow a bad thing, because the presence of more police actually confirms the fact that there is more of a need of police in this neighborhood? Did you understand what you quoted? The author is saying that citations for petty offences (which practically everyone commits with some non-zero frequency, intentionally or not) have a higher chance of affecting future prospects of people in communities where there are more police. the Author isn't saying police are bad in general, rather that the negative effects of police presence are more significant in communities with higher police presence. > So targeted advertising to the needs of people is bad because... I can't even make sense of this one. The message isn't that hard to get, come on. The author is saying (assuming you think like them) that ignorance of suffering is a bad thing. You can say something like "why should I care about people I don't know", and I would understand and even agree to a large part with that statement, but this isn't James Joyce here, you can make sense of it.
- bko 10y ago
  > the Author isn't saying police are bad in general, rather that the negative effects of police presence are more significant in communities with higher police presence. Yes, but are there any positive effects of police presence? Does this outweight the negative effects? > The author is saying (assuming you think like them) that ignorance of suffering is a bad thing So seeing poorly targeted ads tells you that people are suffering? It is the role of corporations to inform you that there are people who have different needs than you?
  - hood_syntax 10y ago
    > Yes, but are there any positive effects of police presence? Does this outweight the negative effects? Certainly I would agree that police have positive effects as well. Your second question really is the crux of the issue, and I can't answer that. > So seeing poorly targeted ads tells you that people are suffering? It is the role of corporations to inform you that there are people who have different needs than you? Suffering was perhaps poor word choice. Obviously it's not the responsibility of some faceless corporation that I be informed about what's going on in the world, but it would be nice if they didn't cut down incidental information the masses might receive so it fits in their little corner of the world.
kaitai 10y ago
You jumped immediately to an assumption of higher crime in the neighborhood, while the article pointed out the existence of white-collar crime in other neighborhoods. An interesting similar practical application was differential sentencing for drug dealing outside versus drug dealing inside. White and black Americans use illegal drugs at essentially the same rate, with white Americans pulling ahead in several categories historically. Suburban Americans more often deal drugs inside houses, while urban Americans are more likely to deal drugs outside than suburban Americans. Due in part to redlining practices, black people are more likely to live in urban areas and white people in suburban areas. So just send a bunch of cops to urban neighborhoods and you'll catch plenty of drug dealers and give them higher sentences, while letting all the white drug dealers go and giving them lower sentences if their behavior is egregious enough that you've got to arrest them. This example doesn't even get into differential treatment for white-collar crime (scamming your investors) vs stealing meat from a grocery store. All those articles we saw on HN about scamming your employees and investors? MotionLoft, 1for.one? Those folks are "just doing business" and most commenters seemed to implicitly condone the scamming behavior -- sure it's unfortunate that it happens but you know business -- while no one here would condone stealing meat from a grocery store, and the meat-lifter would be more likely to get caught since there are cops at the doors of those grocery stores. Do we need cops in the startup scene if scams are happening, though? The bad thing is not targeting "needs," as above, it's targeting perceived needs or exploiting differences in a way that contributes to injustice. I can see how you might not care about targeted advertising that allows you to ignore the lives of the poor -- that's fairly normal and in the US we really rely on it for political stability -- but ads that only show high-cost high-interest low-placement educational options for poor people push out ads from your local community college, targeted ads contribute to the political polarization we're seeing, targeted ads show higher-paying jobs to men than women regardless of qualifications. There is information asymmetry in a lot of these situations and in several ways: if the poor person trying to look up education doesn't already know about the low-cost legit CC and the information is buried after a page of for-profit ads, they may not understand all their options, and just as bad, we who earn a bit more may not know that's happening, because when we look up education to see if our teen could take multivariable calc nearby, we will get the CC right at the top of the page and the for-profit schools, not targeting us, won't show up! And then we can grumble with a clean conscience that poor people are so dumb they can't even look at the first search result...
tsylba 10y ago
> So having more cops in a high-crime neighborhood is somehow a bad thing, because the presence of more police actually confirms the fact that there is more of a need of police in this neighborhood? That's the essence of the Security vs Liberty argument. Yes, on one hand it is a good thing to have more security in an area which needs it; but will you accept say, being filmed 24/7 for more security, with the tradeoff that this same data can be analysed by third parties for any reason whatsoever (ala Minority Report's shopping mall scene). It is not an easy problem and we must decide for ourselves which tradeoffs we will accept, because it is seldom free technology with no strings attached.
- bko 10y ago
  Well this is specifically for high-crime neighborhoods. I would accept less privacy and more policing in high-crime neighborhoods because that reduces the incentives of crime as the perpetrator would be more likely to be caught. When you're talking about marginal benefit (crime rate from 2/100,000 -> 1/100,000), I would be less willing to give up liberties. There is some tipping point, and we can argue where that is, but I still believe in the value of old-fashioned policing to get us to an acceptable risk
merastius 10y ago
I agree with you that for most of these points, the algorithm is unlikely to be less rational or more biased than a human (and the crime-fighting algorithm example doesn't seem to make a good point against the algorithm at all). However, for something like a credit approval algorithm, there is a valid issue of centralisation - if there are problems with the algorithm, a large number of people will be affected negatively by the same algorithm. While humans are biased and imperfect, you can find some who are less biased, or call their manager and see if they can justify themselves, etc. I can imagine situations where it's hard to escape or find recourse when you're faced with an algorithm that's put you in an unfavourable situation. I'm hoping that, given enough data to train on and enough validation to make sure they are working optimally, this will not really be a big issue, but I can understand if some people are concerned about that aspect.

by bko 10y ago

guelo 10y ago

> the presence of more police actually confirms the fact that there is more of a need of police in this neighborhood? If we were to enforce all laws we would need massively more police in every neighborhood. When police are sent to flood a poor neighborhood they end up ticketing poor residents for a bunch of "violations" that they would also find if they went to the rich neighborhoods. But if they ever actually did issue all those ticky-tack tickets in the rich neighborhoods residents would be up in arms contacting their city council members and it would end quickly. In the poor neighborhoods nobody ever hears about it and the residents that can least afford it take the financial hits.

fufubob 10y ago
And how would you prove that hypothesis? With some data and an algorithm. The problem you speak of can be fixed by improving the model, not abandoning it.
burgreblast 10y ago
> they end up ticketing poor residents for a bunch of > "violations" that they would also find if they went > to the rich neighborhoods. I'm not sure what data you have to support this. But check out violent crime rates. Larceny-thefts. Burglary/Property theft. Motor Vehicle Theft. Aggrevated assult. Robbery (mugging, stickups. etc). These very popular crimes aren't so popular in nicer neighborhoods. Rich neighborhoods have their own problems, but let's not appeal to the spirit of lawlessness (cops are bad, rules are bad, unfairly arrested, etc.)
IncRnd 10y ago
There have been zero shootings in the area where I live. More police patrols will not increase the number of shootings. It makes sense for police to be where serious crime occurs.

hood_syntax 10y ago

> So having more cops in a high-crime neighborhood is somehow a bad thing, because the presence of more police actually confirms the fact that there is more of a need of police in this neighborhood? Did you understand what you quoted? The author is saying that citations for petty offences (which practically everyone commits with some non-zero frequency, intentionally or not) have a higher chance of affecting future prospects of people in communities where there are more police. the Author isn't saying police are bad in general, rather that the negative effects of police presence are more significant in communities with higher police presence. > So targeted advertising to the needs of people is bad because... I can't even make sense of this one. The message isn't that hard to get, come on. The author is saying (assuming you think like them) that ignorance of suffering is a bad thing. You can say something like "why should I care about people I don't know", and I would understand and even agree to a large part with that statement, but this isn't James Joyce here, you can make sense of it.

bko 10y ago
> the Author isn't saying police are bad in general, rather that the negative effects of police presence are more significant in communities with higher police presence. Yes, but are there any positive effects of police presence? Does this outweight the negative effects? > The author is saying (assuming you think like them) that ignorance of suffering is a bad thing So seeing poorly targeted ads tells you that people are suffering? It is the role of corporations to inform you that there are people who have different needs than you?
- hood_syntax 10y ago
  > Yes, but are there any positive effects of police presence? Does this outweight the negative effects? Certainly I would agree that police have positive effects as well. Your second question really is the crux of the issue, and I can't answer that. > So seeing poorly targeted ads tells you that people are suffering? It is the role of corporations to inform you that there are people who have different needs than you? Suffering was perhaps poor word choice. Obviously it's not the responsibility of some faceless corporation that I be informed about what's going on in the world, but it would be nice if they didn't cut down incidental information the masses might receive so it fits in their little corner of the world.

kaitai 10y ago

You jumped immediately to an assumption of higher crime in the neighborhood, while the article pointed out the existence of white-collar crime in other neighborhoods. An interesting similar practical application was differential sentencing for drug dealing outside versus drug dealing inside. White and black Americans use illegal drugs at essentially the same rate, with white Americans pulling ahead in several categories historically. Suburban Americans more often deal drugs inside houses, while urban Americans are more likely to deal drugs outside than suburban Americans. Due in part to redlining practices, black people are more likely to live in urban areas and white people in suburban areas. So just send a bunch of cops to urban neighborhoods and you'll catch plenty of drug dealers and give them higher sentences, while letting all the white drug dealers go and giving them lower sentences if their behavior is egregious enough that you've got to arrest them. This example doesn't even get into differential treatment for white-collar crime (scamming your investors) vs stealing meat from a grocery store. All those articles we saw on HN about scamming your employees and investors? MotionLoft, 1for.one? Those folks are "just doing business" and most commenters seemed to implicitly condone the scamming behavior -- sure it's unfortunate that it happens but you know business -- while no one here would condone stealing meat from a grocery store, and the meat-lifter would be more likely to get caught since there are cops at the doors of those grocery stores. Do we need cops in the startup scene if scams are happening, though? The bad thing is not targeting "needs," as above, it's targeting perceived needs or exploiting differences in a way that contributes to injustice. I can see how you might not care about targeted advertising that allows you to ignore the lives of the poor -- that's fairly normal and in the US we really rely on it for political stability -- but ads that only show high-cost high-interest low-placement educational options for poor people push out ads from your local community college, targeted ads contribute to the political polarization we're seeing, targeted ads show higher-paying jobs to men than women regardless of qualifications. There is information asymmetry in a lot of these situations and in several ways: if the poor person trying to look up education doesn't already know about the low-cost legit CC and the information is buried after a page of for-profit ads, they may not understand all their options, and just as bad, we who earn a bit more may not know that's happening, because when we look up education to see if our teen could take multivariable calc nearby, we will get the CC right at the top of the page and the for-profit schools, not targeting us, won't show up! And then we can grumble with a clean conscience that poor people are so dumb they can't even look at the first search result...

tsylba 10y ago

> So having more cops in a high-crime neighborhood is somehow a bad thing, because the presence of more police actually confirms the fact that there is more of a need of police in this neighborhood? That's the essence of the Security vs Liberty argument. Yes, on one hand it is a good thing to have more security in an area which needs it; but will you accept say, being filmed 24/7 for more security, with the tradeoff that this same data can be analysed by third parties for any reason whatsoever (ala Minority Report's shopping mall scene). It is not an easy problem and we must decide for ourselves which tradeoffs we will accept, because it is seldom free technology with no strings attached.

bko 10y ago
Well this is specifically for high-crime neighborhoods. I would accept less privacy and more policing in high-crime neighborhoods because that reduces the incentives of crime as the perpetrator would be more likely to be caught. When you're talking about marginal benefit (crime rate from 2/100,000 -> 1/100,000), I would be less willing to give up liberties. There is some tipping point, and we can argue where that is, but I still believe in the value of old-fashioned policing to get us to an acceptable risk

merastius 10y ago

I agree with you that for most of these points, the algorithm is unlikely to be less rational or more biased than a human (and the crime-fighting algorithm example doesn't seem to make a good point against the algorithm at all). However, for something like a credit approval algorithm, there is a valid issue of centralisation - if there are problems with the algorithm, a large number of people will be affected negatively by the same algorithm. While humans are biased and imperfect, you can find some who are less biased, or call their manager and see if they can justify themselves, etc. I can imagine situations where it's hard to escape or find recourse when you're faced with an algorithm that's put you in an unfavourable situation. I'm hoping that, given enough data to train on and enough validation to make sure they are working optimally, this will not really be a big issue, but I can understand if some people are concerned about that aspect.