3 ms·

> why did your self-driving car hit that pedestrian? It didn't see him/her. Which is the extent of an explanation a human can give you. Try to picture wanting

by Asooka 9y ago

> why did your self-driving car hit that pedestrian? It didn't see him/her. Which is the extent of an explanation a human can give you. Try to picture wanting a person to explain why they didn't see a person coming from the side. We don't know what's happening inside our own heads. That's my point - we're recreating an inherently non-explainable system in software, because we can't explain how it works. The algorithm is so complicated, we can't code it explicitly, so we let the machine figure out the semantic-free equations. All is not bad though, you could always dry-run the algorithm at home, i.e. ask your car "how visible am I with these clothes on", and just not walk outside with low-visibility and low-recognition clothes. Some countries already require pedestrians to wear reflective clothing in the evening to make themselves more visible. I suspect we'll have similar rules for self-driving vehicles: you must wear recognisable clothing when walking through traffic. What I mean about "what if you don't like the explanation" is something else. Let's say you use a deep-learning software to determine sentences. The software spits out "5 years, because this is a black male". Do you accept that? It has been proven effective in many previous cases, can you accept that it has independently discovered a correlation between the person being black and the length of the sentence? Let's say you entirely remove the person's colour and sex from the dataset and re-train, then it will give you shorter sentences than you expect, because every judge previously had subconciously taken all these variables into account. Or it will discover a correlation between wearing e.g. adidas and alcoholism because Slavic people have a higher rate of alcoholism and wear adidas more often than others. Which comes back to my core argument: If you know the acceptable metrics and algorithms, why are you using deep learning? If you don't know the acceptable algorithm, how can you verify the explanation?

gaius 9y ago
It didn't see him/her. Which is the extent of an explanation a human can give you. Try to picture wanting a person to explain why they didn't see a person coming from the side. We don't know what's happening inside our own heads. There is a world of difference between "this individual driver made a human error" and "software that can't reliably see people is driving a million cars now". The developers of the software would need to demonstrate they understood why the software failed in this situation and how they are going to fix it. I suspect we'll have similar rules for self-driving vehicles: you must wear recognisable clothing when walking through traffic Bear in mind that we can't even make cyclists now wear hi-vis or fit lights - making the entire population wear certain things for the benefit of job-destroying technology isn't going to get many votes! If you know the acceptable metrics and algorithms, why are you using deep learning? I disagree, because it's a matter of scale and usefulness. An algorithm that detects pictures of cats can answer the question of "why do you think that's a cat?" with "because it looks like a cat" and that's a perfectly reasonable explanation that a human might give[0]. But the stakes are raised when you start to deal with real people in the real world. "Why did you turn down that loan application" or "why did you add that name to the no-fly list" or "why did you reject that job candidate" need a bit more exposition. That's a self-correcting problem because no-one sane will risk their business on software that "inexplicably" just happens to reject everyone in a protected class because there happens to be some vague correlation between members of that class and some undesirable activity or outcome. [0] Except http://www.bbc.co.uk/news/technology-33347866 http://www.bbc.co.uk/news/technology-33347866 of course
intended 9y ago
> you must wear recognisable clothing when walking through traffic. Which would make Driverless cars a non starter in the real world. Dear God, I can't imagine the poor coder who has to dictate fashion so that car firms can ensure they don't hit people. That addition reneges on the basic expectations people had of driver less cars in the first place, so now why would they have any motivation to let them happen?

by Asooka 9y ago

gaius 9y ago

It didn't see him/her. Which is the extent of an explanation a human can give you. Try to picture wanting a person to explain why they didn't see a person coming from the side. We don't know what's happening inside our own heads. There is a world of difference between "this individual driver made a human error" and "software that can't reliably see people is driving a million cars now". The developers of the software would need to demonstrate they understood why the software failed in this situation and how they are going to fix it. I suspect we'll have similar rules for self-driving vehicles: you must wear recognisable clothing when walking through traffic Bear in mind that we can't even make cyclists now wear hi-vis or fit lights - making the entire population wear certain things for the benefit of job-destroying technology isn't going to get many votes! If you know the acceptable metrics and algorithms, why are you using deep learning? I disagree, because it's a matter of scale and usefulness. An algorithm that detects pictures of cats can answer the question of "why do you think that's a cat?" with "because it looks like a cat" and that's a perfectly reasonable explanation that a human might give[0]. But the stakes are raised when you start to deal with real people in the real world. "Why did you turn down that loan application" or "why did you add that name to the no-fly list" or "why did you reject that job candidate" need a bit more exposition. That's a self-correcting problem because no-one sane will risk their business on software that "inexplicably" just happens to reject everyone in a protected class because there happens to be some vague correlation between members of that class and some undesirable activity or outcome. [0] Except http://www.bbc.co.uk/news/technology-33347866 http://www.bbc.co.uk/news/technology-33347866 of course

intended 9y ago

> you must wear recognisable clothing when walking through traffic. Which would make Driverless cars a non starter in the real world. Dear God, I can't imagine the poor coder who has to dictate fashion so that car firms can ensure they don't hit people. That addition reneges on the basic expectations people had of driver less cars in the first place, so now why would they have any motivation to let them happen?