4 ms·

I think these trolley problems are a waste of everybody's time. Building redundant reliable braking systems will be orders of magnitudes easier than creating a

I think these trolley problems are a waste of everybody's time. Building redundant reliable braking systems will be orders of magnitudes easier than creating a system to fairly and accurately assess who is the best set of people to kill in a disaster scenario.

oska 8y ago
> I think these trolley problems are a waste of everybody's time. I agree. It's an angle that the media love but has little real world applicability.
- mikeash 8y ago
  They’re such contrived scenarios. 99.9999999% of the time the correct response will be obvious, usually “maximum effort braking.”
baroffoos 8y ago
It seems totally useless for self driving cars but it is an interesting view in to human priorities
cscurmudgeon 8y ago
Trolley problems are not about trolleys. Don't take them literally (e.g. Schrodinger's cat).
- TheSmiddy 8y ago
  The original trolley problem of "should you let 5 people die or actively kill 1 to save the 5" has genuine philosophical merit, these however are just so contrived they can't possibly offer any meaningful insight while at the same time planting the seed in the general public that this is the type of problem a self driving car should care about which could cause all kinds of wacky regulations to exist.
  - cscurmudgeon 8y ago
    Agreed.
  - domador 8y ago
    To me that problem had a simple answer: active harm is worse than negligence. To kill someone is a worse action that to let someone die, regardless of the numbers of people involved.
    - sneak 8y ago
      I personally agree with you, however is is extremely important to realize that most of our society does not agree at all, and that that data is hugely relevant.
whyte_mackay 8y ago
I hear this often (that trolley problem is not relevant), but then I discovered that a lot of realistic ML fairness problems can be restated as a trolley problem. You have a classifier for credit assignment (giving a loan, etc.). The classifier is 99% accurate on the entire population. The classifier is 55% accurate on a small minority. You can improve the minority accuracy to 90% at the cost of 0.3% decrease of general accuracy. What do you do? For self-driving: Your accident rate is 0.0001% for the entire population. Your accident rate is 0.0003% for black pedestrians at night. You can allocate more compute/research/resources to equalize the accident rate of black pedestrians at the cost of increasing accident rate for the entire population to 0.00011% (or keeping it constant where you may have seen an improvement if you focused on the general population). What do you do?
- vlovich123 8y ago
  Have the classifier incorporate the identifying characteristics of the minority so that it can push past beyond the 99% limit it's at currently. This is why people are pissed off about the trolly problem. It's all zero-sum hypotheticals when reality is primarily not zero-sum.
  - whyte_mackay 8y ago
    Adding fairness to your models usually incurs a cost which can be measured. You have to choose between max profit or equal opportunity, you can't have both (do you run over the investors or over the minorities?). See here a visualization of trade-off's between global accuracy and fairness: http://research.google.com/bigpicture/attacking-discrimination-in-ml/ http://research.google.com/bigpicture/attacking-discriminati...
  - polkapolka 8y ago
    The identifying characteristics are signal for the minority, but noisy for genpop. Deployment/engineering constraints call for a single model. Realistic scenario.
- joe_the_user 8y ago
  You have a classifier for credit assignment (giving a loan, etc.). The classifier is 99% accurate on the entire population. The classifier is 55% accurate on a small minority. Uh, concerning this hypothetical. I'll admit a scenario of this sort sounds appealing at first blush. But "99%" accuracy rate with credit assignment is transparently absurd if considers it for a second. There is a clear, significant limit to the accuracy that can being assigned to anyone's credit, if credit means "actually repaying". The fundamental uncertainty of the economy guarantees this. The distinction between this ideal (99%-55%) and whatever it might be in reality ( 65%-55%) matters. What's is the system is squeezing a few more percentage points out of data for a large company. And what is the cost of those percentage points? [EDIT: ACTUALLY - the pernicious scenario is a system that isn't not any MORE accurate for any group than any other BUT which is NEGATIVELY biased against one group and POSITIVELY biased against another group. That situation is EASY to get when one unselectively slurps up any data available. The inaccuracy of predictor is a problem for the company, the biasedness of the predictor is a problem of the individuals discriminated against] The situation is that a company really can a total better prediction rate for various desired qualities by using completely biased, unfair markers. (White-skin, went to "a good school", from a wealthy background, dresses well, attractive features...). When one allows "black box optimization" to get those features, what one does is allow the use of these considerations, which all otherwise legally off-limits. Legal strictures against discrimination say that objective measures of black people's ability need to be it, not because other measures never matter but because other measures are unfair, other measures don't consider past discrimination. As a further example, outside of race or gender considerations, some percentage of employees may be forced to care for a sick relative. Maybe that makes them a potentially less effective employee or worse credit risk or whatever. Human evaluators might have values that such questions outside consideration. For an opaque multidimensional analysis, this may a ding - the human user doesn't even know if it's a ding.

oska 8y ago

> I think these trolley problems are a waste of everybody's time. I agree. It's an angle that the media love but has little real world applicability.

mikeash 8y ago
They’re such contrived scenarios. 99.9999999% of the time the correct response will be obvious, usually “maximum effort braking.”

baroffoos 8y ago

It seems totally useless for self driving cars but it is an interesting view in to human priorities

cscurmudgeon 8y ago

Trolley problems are not about trolleys. Don't take them literally (e.g. Schrodinger's cat).

TheSmiddy 8y ago
The original trolley problem of "should you let 5 people die or actively kill 1 to save the 5" has genuine philosophical merit, these however are just so contrived they can't possibly offer any meaningful insight while at the same time planting the seed in the general public that this is the type of problem a self driving car should care about which could cause all kinds of wacky regulations to exist.
- cscurmudgeon 8y ago
  Agreed.
- domador 8y ago
  To me that problem had a simple answer: active harm is worse than negligence. To kill someone is a worse action that to let someone die, regardless of the numbers of people involved.
  - sneak 8y ago
    I personally agree with you, however is is extremely important to realize that most of our society does not agree at all, and that that data is hugely relevant.

whyte_mackay 8y ago

I hear this often (that trolley problem is not relevant), but then I discovered that a lot of realistic ML fairness problems can be restated as a trolley problem. You have a classifier for credit assignment (giving a loan, etc.). The classifier is 99% accurate on the entire population. The classifier is 55% accurate on a small minority. You can improve the minority accuracy to 90% at the cost of 0.3% decrease of general accuracy. What do you do? For self-driving: Your accident rate is 0.0001% for the entire population. Your accident rate is 0.0003% for black pedestrians at night. You can allocate more compute/research/resources to equalize the accident rate of black pedestrians at the cost of increasing accident rate for the entire population to 0.00011% (or keeping it constant where you may have seen an improvement if you focused on the general population). What do you do?

vlovich123 8y ago
Have the classifier incorporate the identifying characteristics of the minority so that it can push past beyond the 99% limit it's at currently. This is why people are pissed off about the trolly problem. It's all zero-sum hypotheticals when reality is primarily not zero-sum.
- whyte_mackay 8y ago
  Adding fairness to your models usually incurs a cost which can be measured. You have to choose between max profit or equal opportunity, you can't have both (do you run over the investors or over the minorities?). See here a visualization of trade-off's between global accuracy and fairness: http://research.google.com/bigpicture/attacking-discrimination-in-ml/ http://research.google.com/bigpicture/attacking-discriminati...
- polkapolka 8y ago
  The identifying characteristics are signal for the minority, but noisy for genpop. Deployment/engineering constraints call for a single model. Realistic scenario.
joe_the_user 8y ago
You have a classifier for credit assignment (giving a loan, etc.). The classifier is 99% accurate on the entire population. The classifier is 55% accurate on a small minority. Uh, concerning this hypothetical. I'll admit a scenario of this sort sounds appealing at first blush. But "99%" accuracy rate with credit assignment is transparently absurd if considers it for a second. There is a clear, significant limit to the accuracy that can being assigned to anyone's credit, if credit means "actually repaying". The fundamental uncertainty of the economy guarantees this. The distinction between this ideal (99%-55%) and whatever it might be in reality ( 65%-55%) matters. What's is the system is squeezing a few more percentage points out of data for a large company. And what is the cost of those percentage points? [EDIT: ACTUALLY - the pernicious scenario is a system that isn't not any MORE accurate for any group than any other BUT which is NEGATIVELY biased against one group and POSITIVELY biased against another group. That situation is EASY to get when one unselectively slurps up any data available. The inaccuracy of predictor is a problem for the company, the biasedness of the predictor is a problem of the individuals discriminated against] The situation is that a company really can a total better prediction rate for various desired qualities by using completely biased, unfair markers. (White-skin, went to "a good school", from a wealthy background, dresses well, attractive features...). When one allows "black box optimization" to get those features, what one does is allow the use of these considerations, which all otherwise legally off-limits. Legal strictures against discrimination say that objective measures of black people's ability need to be it, not because other measures never matter but because other measures are unfair, other measures don't consider past discrimination. As a further example, outside of race or gender considerations, some percentage of employees may be forced to care for a sick relative. Maybe that makes them a potentially less effective employee or worse credit risk or whatever. Human evaluators might have values that such questions outside consideration. For an opaque multidimensional analysis, this may a ding - the human user doesn't even know if it's a ding.