4 ms·

Is this for anyone or is it just posturing? I mean anyone doing ML knows that if you train an algorithm on a biased dataset that it will, surprise, learn to be

Is this for anyone or is it just posturing? I mean anyone doing ML knows that if you train an algorithm on a biased dataset that it will, surprise, learn to be biased. It's not at all inherent or restricted to unfair treatment of any group. It's a more general problem. By focusing on "fairness" you are basically overfitting the larger problem of not selecting good datasets or not having good data.

nkozyra 8y ago
Huh? Humans make the conscious decision to override instinctual and observational impulses all the time in response to empathy and emotion. It is a huge foundational factor in society. Why would we NOT want autonomous agents to have the same capacity? There's a reason things like The Trolley Problem come up in AI courses: our agents have to play reasonably in a human world.
- sokoloff 8y ago
  I find the classic Trolley Problem interesting mostly because it exposes the extremely limited capacity that humans have to act rationally. I would expect a trivial ML/AI implementation to "correctly" solve the Trolley Problem and I would not want to consciously program our future AI systems to suffer from the same limitations that cause humans to struggle with the Trolley Problem. By programming them this way (avoiding programming in human frailties for the purpose of better "fitting in"), I think we can more quickly expose and correct certain areas where us bags of mostly salt water act suboptimally.
  - nkozyra 8y ago
    The whole point of the trolley problem is there is no correct solution. It's a negative sum game. It's an ethical problem that has no solution. There is also no guarantee that a machine would kill one in lieu of five. Why? Training data. The bigger point I'm making is people think AI acts in some form of posthuman reason. At this time no such thing exists: decisions are still bound to human bias, and we should at least consider optimizing with regard to the rules of a modern society.
    - sokoloff 8y ago
      My intentionally off-hand use of "correctly" was meant to indicate a belief that killing one innocent is strictly better than killing five and so if a machine is faced with choosing exactly one of those, the problem boils down to a comparison and a conditional jump, IMO. Humans are constrained by the knowledge that they would dread for the rest of their life the knowledge that they optionally and intentionally took an overt action which resulted in the death of an innocent person (to save five, but they would know they killed one when they could have done nothing and had "clean hands"). Machines don't need to suffer from this. I would hope that if we program safety-controlling systems that we wouldn't replace the compare and conditional-jump with "halt and catch fire".
      - nkozyra 8y ago
        No offense here, but none of that has anything to do with the dilemma. Human response would be biased in myriad ways. Agent response would be biased in myriad way. There is no correct solution. The lasting impact on a person (and lack thereof in an agent) has no bearing on the decision.
        sokoloff 8y ago
        Really? Why is the original/classic trolley problem in any way interesting if the future weight of the decision on the human has no bearing? (That's a serious question.) In the problem statement, an actor has control over whether this 1 person or those 5 people die. If they choose red, then 1. If they choose blue, then 5. (Failing to make a choice is choosing blue.) If after playing the scenario, they knew someone would use a Men In Black pen on them to completely forget what happened, is there a rational argument against choosing red? (I realize this might read like trolling, but I genuinely don't find the original problem a difficult one if you eliminate the human guilt aspect.)
        carapat_virulat 8y ago
        If you choose to switch the track to kill 1 innocent to save 5, you open the door to things like a society where 1 innocent healthy person is killed to harvest the organs and save 5 innocents. The calculation is the same.
  - chongli 8y ago
    So let's suppose you were riding in your self-driving car and the AI in charge of it encountered a trolley problem: The brakes had suddenly failed approaching an intersection and the car had the choice of either running into a group of five pedestrians crossing the street or slamming into the concrete wall to your right, killing only you. Which is the correct answer? Now suppose it was not you in the car but your wife and two daughters. Would you want your car, the car you bought and paid for, to sacrifice your family to save the lives of five strangers? What if it were only four strangers? Or ten? There is no right answer. But as humans we're biased to favour, if not our own lives, the lives of our closest family members over strangers. I, personally, would never buy a car if I knew it would make the choice to sacrifice my family to save strangers.
    - sokoloff 8y ago
      Yes, I agree humans are selfish. Given the enormous number of situations where personal or in-group selfishness would confer a survival and propagation advantage, it would be surprising if we didn't have it. I hope we don't take explicit pains and effort to encode this into ML/AI systems. Concretely, the car should take the course of action to minimize the global harm based on the knowledge it has. If that's killing me in the event of a brake failure, so be it. The spectacularly unlikely probability of that is outweighed by the million more times when it and others like it will make a less selfish decision than today's human drivers. > I, personally, would never buy a car if I knew it would make the choice to sacrifice my family to save strangers. The point I find interesting here is "this consumer preference, if common, confers a propagation advantage to systems which behave selfishly".
      - justtopost 8y ago
        But should we be deprived of our instinct? Should we not get to apply our own personal ethics to our lives in the future, for the greater good? Its not a welcome encroachment on free will, even if it may be net positive from an objective view, removing the sense of agencey from humanity may end up hurting us worse.
- AndrewKemendo 8y ago
  Humans make the conscious decision to override instinctual and observational impulses all the time in response to empathy and emotion. This statement is misleading. "Instinctual and observational impulses" is poorly defined, but the low latency cognitive systems I think you are describing (such as object categorization) are inputs to higher executive functioning systems which determine behavior. The fact of and ability to "override" generalizes too much about how these low latency systems are prioritized a-priori. In other words, some of them seem to be able to be able to be re-prioritized fairly easily (positive/negative response to specific olfactory stimuli), while others approach impossible without motor system intervention (Not reacting to severe pain stimuli). It's categorically misleading to separate these from all other functions of Perceive > Process > Respond as all decisions are all built on the same reinforcement learning foundations.
  - nkozyra 8y ago
    You've created a strawman here but I applaud your argument against it.
    - AndrewKemendo 8y ago
      Not sure where the strawman is. I'm trying to describe that the cognitive process you think exists, doesn't actually exist the way you seem to think it does.

nkozyra 8y ago

Huh? Humans make the conscious decision to override instinctual and observational impulses all the time in response to empathy and emotion. It is a huge foundational factor in society. Why would we NOT want autonomous agents to have the same capacity? There's a reason things like The Trolley Problem come up in AI courses: our agents have to play reasonably in a human world.

sokoloff 8y ago
I find the classic Trolley Problem interesting mostly because it exposes the extremely limited capacity that humans have to act rationally. I would expect a trivial ML/AI implementation to "correctly" solve the Trolley Problem and I would not want to consciously program our future AI systems to suffer from the same limitations that cause humans to struggle with the Trolley Problem. By programming them this way (avoiding programming in human frailties for the purpose of better "fitting in"), I think we can more quickly expose and correct certain areas where us bags of mostly salt water act suboptimally.
- nkozyra 8y ago
  The whole point of the trolley problem is there is no correct solution. It's a negative sum game. It's an ethical problem that has no solution. There is also no guarantee that a machine would kill one in lieu of five. Why? Training data. The bigger point I'm making is people think AI acts in some form of posthuman reason. At this time no such thing exists: decisions are still bound to human bias, and we should at least consider optimizing with regard to the rules of a modern society.
  - sokoloff 8y ago
    My intentionally off-hand use of "correctly" was meant to indicate a belief that killing one innocent is strictly better than killing five and so if a machine is faced with choosing exactly one of those, the problem boils down to a comparison and a conditional jump, IMO. Humans are constrained by the knowledge that they would dread for the rest of their life the knowledge that they optionally and intentionally took an overt action which resulted in the death of an innocent person (to save five, but they would know they killed one when they could have done nothing and had "clean hands"). Machines don't need to suffer from this. I would hope that if we program safety-controlling systems that we wouldn't replace the compare and conditional-jump with "halt and catch fire".
    - nkozyra 8y ago
      No offense here, but none of that has anything to do with the dilemma. Human response would be biased in myriad ways. Agent response would be biased in myriad way. There is no correct solution. The lasting impact on a person (and lack thereof in an agent) has no bearing on the decision.
      - sokoloff 8y ago
        Really? Why is the original/classic trolley problem in any way interesting if the future weight of the decision on the human has no bearing? (That's a serious question.) In the problem statement, an actor has control over whether this 1 person or those 5 people die. If they choose red, then 1. If they choose blue, then 5. (Failing to make a choice is choosing blue.) If after playing the scenario, they knew someone would use a Men In Black pen on them to completely forget what happened, is there a rational argument against choosing red? (I realize this might read like trolling, but I genuinely don't find the original problem a difficult one if you eliminate the human guilt aspect.)
        carapat_virulat 8y ago
        If you choose to switch the track to kill 1 innocent to save 5, you open the door to things like a society where 1 innocent healthy person is killed to harvest the organs and save 5 innocents. The calculation is the same.
- chongli 8y ago
  So let's suppose you were riding in your self-driving car and the AI in charge of it encountered a trolley problem: The brakes had suddenly failed approaching an intersection and the car had the choice of either running into a group of five pedestrians crossing the street or slamming into the concrete wall to your right, killing only you. Which is the correct answer? Now suppose it was not you in the car but your wife and two daughters. Would you want your car, the car you bought and paid for, to sacrifice your family to save the lives of five strangers? What if it were only four strangers? Or ten? There is no right answer. But as humans we're biased to favour, if not our own lives, the lives of our closest family members over strangers. I, personally, would never buy a car if I knew it would make the choice to sacrifice my family to save strangers.
  - sokoloff 8y ago
    Yes, I agree humans are selfish. Given the enormous number of situations where personal or in-group selfishness would confer a survival and propagation advantage, it would be surprising if we didn't have it. I hope we don't take explicit pains and effort to encode this into ML/AI systems. Concretely, the car should take the course of action to minimize the global harm based on the knowledge it has. If that's killing me in the event of a brake failure, so be it. The spectacularly unlikely probability of that is outweighed by the million more times when it and others like it will make a less selfish decision than today's human drivers. > I, personally, would never buy a car if I knew it would make the choice to sacrifice my family to save strangers. The point I find interesting here is "this consumer preference, if common, confers a propagation advantage to systems which behave selfishly".
    - justtopost 8y ago
      But should we be deprived of our instinct? Should we not get to apply our own personal ethics to our lives in the future, for the greater good? Its not a welcome encroachment on free will, even if it may be net positive from an objective view, removing the sense of agencey from humanity may end up hurting us worse.
AndrewKemendo 8y ago
Humans make the conscious decision to override instinctual and observational impulses all the time in response to empathy and emotion. This statement is misleading. "Instinctual and observational impulses" is poorly defined, but the low latency cognitive systems I think you are describing (such as object categorization) are inputs to higher executive functioning systems which determine behavior. The fact of and ability to "override" generalizes too much about how these low latency systems are prioritized a-priori. In other words, some of them seem to be able to be able to be re-prioritized fairly easily (positive/negative response to specific olfactory stimuli), while others approach impossible without motor system intervention (Not reacting to severe pain stimuli). It's categorically misleading to separate these from all other functions of Perceive > Process > Respond as all decisions are all built on the same reinforcement learning foundations.
- nkozyra 8y ago
  You've created a strawman here but I applaud your argument against it.
  - AndrewKemendo 8y ago
    Not sure where the strawman is. I'm trying to describe that the cognitive process you think exists, doesn't actually exist the way you seem to think it does.