4 ms·

I don't follow you. When do you start with biased data? Furthermore, wouldn't a advertisement system that somehow progresses to an increasingly erroneous view b

I don't follow you. When do you start with biased data? Furthermore, wouldn't a advertisement system that somehow progresses to an increasingly erroneous view be broken anyway?

rm_-rf_slash 10y ago
If historic and contemporary data assumes American blacks live in the city and whites live in the suburbs, data alone will not "know" the histories of race-based discrimination. So if it advertises apartments to blacks and houses to whites, it does not "understand" that it is propagating a stereotype. A human has to intervene.
- TruthAndDare 10y ago
  The point of the algorithm is simply to choose advertisements as successfully as possible.
ubernostrum 10y ago
Current distributions of property, wealth, access to services, etc. are all rooted in discriminatory laws and policies of the (not always distant) past. Which creates a risk that blindly pointing an algorithm at these data sets will result in an algorithm which assumes these are the expected/natural distributions and will optimize to perpetuate them, essentially becoming an algorithmic implementation of the prior discriminatory laws and policies. Thus, unless the goal is to perpetuate the effects of discrimination, algorithms need to be carefully set up and trained to understand that "this isn't how things are supposed to be".
- selectron 10y ago
  The algorithm just tries to model the world as it actually is, not as it is supposed to be. The algorithm cares nothing about uncomfortable truths. The algorithm should not be set up and trained to reflect a wrong model of reality, but rather you should consider if you want to use an algorithm based on reality or one based on how you want the world to be.
  - Joof 10y ago
    Fair enough. In many cases we would prefer to model it how we want the world to be in order to overcome human misconceptions that have caused the world to be as it is currently.
    - prodigal_erik 10y ago
      Society would have to compensate me for turning a blind eye to today's reality, because otherwise I'm giving myself a competitive disadvantage vs. people with accurate tools.
  - cortesoft 10y ago
    It really depends on what your algorithm is determining.
  - dwaltrip 10y ago
    Yes, I'm glad we are in agreement. We shouldn't limit ourselves to current state of the world. We should design systems that help us get closer to an improved world.
  - beat 10y ago
    This isn't hard to understand. Suppose crime is equal in two neighborhoods, but the police patrol one neighborhood 500% more than the other. What would the result of the algorithm be? Five times more arrests in one neighborhood than another. Now, take that model and apply a really crude analysis. You would conclude that there is five times more crime in the more heavily patrolled neighborhood. Your algorithm, applied to policy, suggests that crime could be controlled better by patrolling the "bad" neighborhood ten times as much, rather than five times as much. Now, consider the social result. The "high crime neighborhood" then sees its property values drop, and more homes sold into the rental market. This attracts poorer, rougher people who can't afford the "good" neighborhood. Now crime actually goes up, reinforcing the idea that the bad neighborhood is high-crime. Now, imagine we've been doing this to those two neighborhoods for generations. What are the likely results of your algorithm? There's your uncomfortable truth.
pdkl95 10y ago
> When do you start with biased data? In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6. "What are you doing?", asked Minsky. "I am training a randomly wired neural net to play Tic-Tac-Toe" Sussman replied. "Why is the net wired randomly?", asked Minsky. "I do not want it to have any preconceptions of how to play", Sussman said. Minsky then shut his eyes. "Why do you close your eyes?", Sussman asked his teacher. "So that the room will be empty." At that moment, Sussman was enlightened. (from: http://catb.org/jargon/html/koans.html#id3141241 http://catb.org/jargon/html/koans.html#id3141241 ) There is always bias (and other types of error) in data. There is even bias in the choice of which data to use and the type of analysis to perform. If you think that this isn't a problem, you should really read about practices like "redlining"[1]. For many decades segregation was (and still is) enforced by opaque "loan approval" methods that just happened to always deny loans to blacks. [1] http://www.theatlantic.com/magazine/archive/2014/06/the-case-for-reparations/361631/ http://www.theatlantic.com/magazine/archive/2014/06/the-case...

I don't follow you. When do you start with biased data? Furthermore, wouldn't a advertisement system that somehow progresses to an increasingly erroneous view be broken anyway?

rm_-rf_slash 10y ago

If historic and contemporary data assumes American blacks live in the city and whites live in the suburbs, data alone will not "know" the histories of race-based discrimination. So if it advertises apartments to blacks and houses to whites, it does not "understand" that it is propagating a stereotype. A human has to intervene.

TruthAndDare 10y ago
The point of the algorithm is simply to choose advertisements as successfully as possible.

ubernostrum 10y ago

Current distributions of property, wealth, access to services, etc. are all rooted in discriminatory laws and policies of the (not always distant) past. Which creates a risk that blindly pointing an algorithm at these data sets will result in an algorithm which assumes these are the expected/natural distributions and will optimize to perpetuate them, essentially becoming an algorithmic implementation of the prior discriminatory laws and policies. Thus, unless the goal is to perpetuate the effects of discrimination, algorithms need to be carefully set up and trained to understand that "this isn't how things are supposed to be".

selectron 10y ago
The algorithm just tries to model the world as it actually is, not as it is supposed to be. The algorithm cares nothing about uncomfortable truths. The algorithm should not be set up and trained to reflect a wrong model of reality, but rather you should consider if you want to use an algorithm based on reality or one based on how you want the world to be.
- Joof 10y ago
  Fair enough. In many cases we would prefer to model it how we want the world to be in order to overcome human misconceptions that have caused the world to be as it is currently.
  - prodigal_erik 10y ago
    Society would have to compensate me for turning a blind eye to today's reality, because otherwise I'm giving myself a competitive disadvantage vs. people with accurate tools.
- cortesoft 10y ago
  It really depends on what your algorithm is determining.
- dwaltrip 10y ago
  Yes, I'm glad we are in agreement. We shouldn't limit ourselves to current state of the world. We should design systems that help us get closer to an improved world.
- beat 10y ago
  This isn't hard to understand. Suppose crime is equal in two neighborhoods, but the police patrol one neighborhood 500% more than the other. What would the result of the algorithm be? Five times more arrests in one neighborhood than another. Now, take that model and apply a really crude analysis. You would conclude that there is five times more crime in the more heavily patrolled neighborhood. Your algorithm, applied to policy, suggests that crime could be controlled better by patrolling the "bad" neighborhood ten times as much, rather than five times as much. Now, consider the social result. The "high crime neighborhood" then sees its property values drop, and more homes sold into the rental market. This attracts poorer, rougher people who can't afford the "good" neighborhood. Now crime actually goes up, reinforcing the idea that the bad neighborhood is high-crime. Now, imagine we've been doing this to those two neighborhoods for generations. What are the likely results of your algorithm? There's your uncomfortable truth.

pdkl95 10y ago

> When do you start with biased data? In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6. "What are you doing?", asked Minsky. "I am training a randomly wired neural net to play Tic-Tac-Toe" Sussman replied. "Why is the net wired randomly?", asked Minsky. "I do not want it to have any preconceptions of how to play", Sussman said. Minsky then shut his eyes. "Why do you close your eyes?", Sussman asked his teacher. "So that the room will be empty." At that moment, Sussman was enlightened. (from: http://catb.org/jargon/html/koans.html#id3141241 http://catb.org/jargon/html/koans.html#id3141241 ) There is always bias (and other types of error) in data. There is even bias in the choice of which data to use and the type of analysis to perform. If you think that this isn't a problem, you should really read about practices like "redlining"[1]. For many decades segregation was (and still is) enforced by opaque "loan approval" methods that just happened to always deny loans to blacks. [1] http://www.theatlantic.com/magazine/archive/2014/06/the-case-for-reparations/361631/ http://www.theatlantic.com/magazine/archive/2014/06/the-case...