4 ms·

If the car has the ability to make these kinds of distinctions in such a simple scenario, surely there are more options than two. "Moral?" It smells like euge

If the car has the ability to make these kinds of distinctions in such a simple scenario, surely there are more options than two. "Moral?" It smells like eugenics to me: some bizarre, Ivy League technocratic posthumanism. Somehow the value of a life is determined by age, sex and profession? Says who? The people who are programming dataset via https? This is collapsing nuanced spiritual and ethical intuition onto an extremely narrow, low dimensional set of parameters. What's the premise? This triggers in me an imagination of naive, optimistic, well-adjusted Germans in the 1930s. I know this was probably created with good intentions, but the premise does not match the research question. The premise is "morality". Yet its asking me to rank the value of human life based on presumptuous, superficial categories. Is "the moral machine" going to also decide which births have more utility? Which countries to send aide to? Who should have access to educational opportunities or quality food? Based on low dimensional datasets such as this one?

JoshTriplett 10y ago
Agreed. This has too many axes of differences, and doesn't have sufficiently careful control and consistency to determine which of them are being consistently ignored. It looks like they were trying to see if people place differing amounts of value on different human lives, but in the process of doing so, they made ridiculously strange value judgments. "Athlete"? "Executive"? "Large"? Why should any of those matter? We're talking about human lives.
- hexane360 10y ago
  I think part of their intent was to show biases in judgement. The small sample size really hindered this though. Apparently I 100% preferred old people to young people, even though I didn't consider age in my decisions at all. They do have a little disclaimer on the results screen about the sample size though.
  - macawfish 10y ago
    The bias is generated by the lack of real choice in responses. They are offering an unrepresentative range of choices, then going and telling you that your choices are biased. They're not showing existing bias, they're coercing it out of participants. This thing irritated me so bad that I couldn't bring myself to answer a single question. I can't stand the idea that my this study is actually happening. If the results are published and receive non-critical media attention, I'm gonna be irate.
    - esfandia 10y ago
      But then you'd need to answer 100 or more test questions instead of 13. This is necessarily simplistic given how much involvement they can ask of participants. Also, I don't think they're implying anything nefarious about the resulting biases shown in the final results. For sure a lot of it has to do with the small sample size of the questions. I didn't run the tests twice, but I imagine there is some randomness involved in the way they are generated. If there are any heuristics at play, the results will indeed show them (in my case there were enough tests to recover the fact that I preferred saving passengers, preferred non-intervention, and preferred saving humans over pets). But it will also come up with some gibberish/noise due to the small sample size.
      - JoshTriplett 10y ago
        > But then you'd need to answer 100 or more test questions instead of 13. Or they could design questions designed to more definitively separate different hypotheses in fewer questions.
      - macawfish 10y ago
        It doesn't matter how big the sample size is if the questions themselves are biased.
  - fela 10y ago
    On a single participant yes, but their real goal, I presume, is to aggregate the data and then they will be able to reach more concrete conclusions.
macawfish 10y ago
If anything, it'd be more "moral" in situations like this for the "car" to choose a random sacrifice than to attempt some half-assed wanna-be God crap like this. I'm honestly appalled at this right now. It's not like people haven't seen this sort of thing coming down the line. It's just surreal to watch it arrive.
OilDerek 10y ago
Yep. Going in, for some reason I thought this would be a kind of logic puzzle, e.g. given these pedestrians/road hazards, nagivate to safety. The first scenario I saw was "hit and kill three joggers, or hit and kill three fat people?" That's just...morally tone deaf and sick on about half a dozen levels.
johanneskanybal 10y ago
In that sense I guess the thought experiment worked. Perhaps not as intended though. It showed the absurdum in pretending a car can make moral choices.
fela 10y ago
We do make moral choices, and there are rules and heuristics we use, they might be quite complicated, and they might not be what we think they are, but I think nonetheless that it should be possible to come close to predicting a human moral decision making quite well by using an accurate enough model. And as autonomous vehicles will have to make decisions that have moral implications, they better do so in a way that humans will be happy with. I think this is an important area of research. This won't mean a machines will have morals of his own, whatever that means, but that they should do what (most?) humans would consider morally right. And what do humans consider morally right? Well that is exactly what we should try to find out.
koliber 10y ago
I agree that categorizing morality into buckets seems strange. However, over a large sample, surveys with even these very limited, artificial choices can paint a surprisingly accurate picture of the nuanced, fine-tuned moral compass of a society. I would like to have automated systems use logic that reflects the morals and values of the society in which they operate. I don't know how to measure those accurately. These sorts of exercises seem like a good start.
- macawfish 10y ago
  I agree that people gotta start somewhere. And I agree that it might be possible to paint useful models of human morality with small sets of parameters... Just not the way this study is set up. Not with the parameters they're measuring, and definitely not out of the logical presumptions of the experiment. I am presented with the choice that either 4 women must die or 4 men must die. For me, it would be more "moral" in this case, for the computer to choose randomly, rather than to attempt some shallow, eugenically judgemental "moral" logic. I'm also aware that these kinds of rules, regardless of their "morality", can be gamed. Randomization increases the risk for people considering playing games. This adds more weight to my conviction that, if some of these false dilemmas really did present themselves to a machine, in real life, that randomization must be an option. How does this moral logic map to this survey? It doesn't map, Not one single bit. That irritates me, because if I were to click through this survey, using eenie meenie miney moe in cases I felt that randomization would be more moral, it would be all but lost in the error. The MIT students would go on CBS morning media and talk about all the bias they measured in my choices of who to murder. But their data would be totally polluted by the way their study discounted moral logics outside of their parameter set. And important parts of my moral reasoning would be lost in the error bars. What's more, my conviction about the necessity of randomization is just just one of a huge variety of moral considerations that are inherent to peoples' sense of morality. Hopefully the study is considering these kinds of things and they have some clever way mathematical of extracting useful information out of this data. For example, hopefully they are measuring the number of people who visited these pages but refused to make a choice.

JoshTriplett 10y ago

Agreed. This has too many axes of differences, and doesn't have sufficiently careful control and consistency to determine which of them are being consistently ignored. It looks like they were trying to see if people place differing amounts of value on different human lives, but in the process of doing so, they made ridiculously strange value judgments. "Athlete"? "Executive"? "Large"? Why should any of those matter? We're talking about human lives.

hexane360 10y ago
I think part of their intent was to show biases in judgement. The small sample size really hindered this though. Apparently I 100% preferred old people to young people, even though I didn't consider age in my decisions at all. They do have a little disclaimer on the results screen about the sample size though.
- macawfish 10y ago
  The bias is generated by the lack of real choice in responses. They are offering an unrepresentative range of choices, then going and telling you that your choices are biased. They're not showing existing bias, they're coercing it out of participants. This thing irritated me so bad that I couldn't bring myself to answer a single question. I can't stand the idea that my this study is actually happening. If the results are published and receive non-critical media attention, I'm gonna be irate.
  - esfandia 10y ago
    But then you'd need to answer 100 or more test questions instead of 13. This is necessarily simplistic given how much involvement they can ask of participants. Also, I don't think they're implying anything nefarious about the resulting biases shown in the final results. For sure a lot of it has to do with the small sample size of the questions. I didn't run the tests twice, but I imagine there is some randomness involved in the way they are generated. If there are any heuristics at play, the results will indeed show them (in my case there were enough tests to recover the fact that I preferred saving passengers, preferred non-intervention, and preferred saving humans over pets). But it will also come up with some gibberish/noise due to the small sample size.
    - JoshTriplett 10y ago
      > But then you'd need to answer 100 or more test questions instead of 13. Or they could design questions designed to more definitively separate different hypotheses in fewer questions.
    - macawfish 10y ago
      It doesn't matter how big the sample size is if the questions themselves are biased.
- fela 10y ago
  On a single participant yes, but their real goal, I presume, is to aggregate the data and then they will be able to reach more concrete conclusions.

macawfish 10y ago

If anything, it'd be more "moral" in situations like this for the "car" to choose a random sacrifice than to attempt some half-assed wanna-be God crap like this. I'm honestly appalled at this right now. It's not like people haven't seen this sort of thing coming down the line. It's just surreal to watch it arrive.

OilDerek 10y ago

Yep. Going in, for some reason I thought this would be a kind of logic puzzle, e.g. given these pedestrians/road hazards, nagivate to safety. The first scenario I saw was "hit and kill three joggers, or hit and kill three fat people?" That's just...morally tone deaf and sick on about half a dozen levels.

johanneskanybal 10y ago

In that sense I guess the thought experiment worked. Perhaps not as intended though. It showed the absurdum in pretending a car can make moral choices.

fela 10y ago

We do make moral choices, and there are rules and heuristics we use, they might be quite complicated, and they might not be what we think they are, but I think nonetheless that it should be possible to come close to predicting a human moral decision making quite well by using an accurate enough model. And as autonomous vehicles will have to make decisions that have moral implications, they better do so in a way that humans will be happy with. I think this is an important area of research. This won't mean a machines will have morals of his own, whatever that means, but that they should do what (most?) humans would consider morally right. And what do humans consider morally right? Well that is exactly what we should try to find out.

koliber 10y ago

I agree that categorizing morality into buckets seems strange. However, over a large sample, surveys with even these very limited, artificial choices can paint a surprisingly accurate picture of the nuanced, fine-tuned moral compass of a society. I would like to have automated systems use logic that reflects the morals and values of the society in which they operate. I don't know how to measure those accurately. These sorts of exercises seem like a good start.

macawfish 10y ago
I agree that people gotta start somewhere. And I agree that it might be possible to paint useful models of human morality with small sets of parameters... Just not the way this study is set up. Not with the parameters they're measuring, and definitely not out of the logical presumptions of the experiment. I am presented with the choice that either 4 women must die or 4 men must die. For me, it would be more "moral" in this case, for the computer to choose randomly, rather than to attempt some shallow, eugenically judgemental "moral" logic. I'm also aware that these kinds of rules, regardless of their "morality", can be gamed. Randomization increases the risk for people considering playing games. This adds more weight to my conviction that, if some of these false dilemmas really did present themselves to a machine, in real life, that randomization must be an option. How does this moral logic map to this survey? It doesn't map, Not one single bit. That irritates me, because if I were to click through this survey, using eenie meenie miney moe in cases I felt that randomization would be more moral, it would be all but lost in the error. The MIT students would go on CBS morning media and talk about all the bias they measured in my choices of who to murder. But their data would be totally polluted by the way their study discounted moral logics outside of their parameter set. And important parts of my moral reasoning would be lost in the error bars. What's more, my conviction about the necessity of randomization is just just one of a huge variety of moral considerations that are inherent to peoples' sense of morality. Hopefully the study is considering these kinds of things and they have some clever way mathematical of extracting useful information out of this data. For example, hopefully they are measuring the number of people who visited these pages but refused to make a choice.