5 ms·

Random thought: the more deep learning is used in training, the less humans will be able to retroactively explain decisions; this surely has liability implicati

Random thought: the more deep learning is used in training, the less humans will be able to retroactively explain decisions; this surely has liability implications

bobcostas55 10y ago
Perhaps NNs need functionality to come up with fake rationalizations for their decisions just like the human brain does.
- worldsayshi 10y ago
  I'm thinking that this idea about "fake rationalization" is a bit backwards. We say that many actions happen before we are aware of it or have a rationalization for them but becoming aware of why we did something after the fact doesn't mean that "you" weren't part of the decision. Aren't your reflexes a part of "you"?
  - bobcostas55 10y ago
    Of course, but nobody is going to say "my reflexes did it". We come up with stories, lines of argumentation, etc. Even though these are "ex post" we often manage to convince ourselves... In Homer's time the Greeks viewed the unconscious as an external (divine) force, the most famous example being Telemachus's sneeze. In a weird way this feels more intellectually honest than the thing we do.
    - peteretep 10y ago
      > we often manage to convince ourselves In this case, presumably the trick is to convince a jury.
wavefunction 10y ago
I don't see how this is true. With a machine system you would at least have the ability to log the mathematical operations and results, which would be complex, sure. But more complex than figuring out if another person is lying about a very nuanced and opaque decision they made?
- feel_the_need 10y ago
  The scope of your question is too large. Do not compare the complexity of AI to the human brain. Instead compare AI to traditional robotics. In expert systems rules are laid out and logic chains can be followed. AI is a black box of a billion random weighted nodes. The system does not care about 'why' when reducing loss.
catwell 10y ago
Yes. This is why https://news.ycombinator.com/item?id=13111516 https://news.ycombinator.com/item?id=13111516 was on the front page last month.
javiramos 10y ago
NuTonomy has an autonomous driving technology based on formal logic [1]. It seems like formal logic is a better approach for retroactively explaining software decisions. [1] http://spectrum.ieee.org/transportation/self-driving/after-mastering-singapores-streets-nutonomys-robotaxis-are-poised-to-take-on-new-cities http://spectrum.ieee.org/transportation/self-driving/after-m...
- Fricken 10y ago
  Google too uses plenty of formal logic in their autonomous OS, it's a frankenstein of various machine learning techniques. Narrow pattern recognition alone, while powerful, is unlikely to ever be suitable for things like negotiating busy four way stops. There is the presumption that all you gotta do is gather enough training data and shazam!, you got a self driving car, but that's bollocks. There is a tremendous amount of elbow grease that foes into developing and validating an AV to six sigma reliability. Six sigma might not even be good enough for something as safety-critical as a driverless car.
  - rthille 10y ago
    It doesn't need six-sigma, just to be significantly better than humans, and for the manufacturer to take on the liability.
    - Fricken 10y ago
      In strictly rational terms It has to be better than 1 fatality for every 100,000,000 miles driven, but public opinion isn't exactly rational so it probably needs to be even better than that.
      - xorxornop 10y ago
        It'll be that way at first, yeah, but once it becomes obvious how much better the AI drivers are, it will start to become irresponsible to not use it
whistlerbrk 10y ago
I feel like this is very actively being addressed, for example by visualization of intermediate features.

Random thought: the more deep learning is used in training, the less humans will be able to retroactively explain decisions; this surely has liability implications

bobcostas55 10y ago

Perhaps NNs need functionality to come up with fake rationalizations for their decisions just like the human brain does.

worldsayshi 10y ago
I'm thinking that this idea about "fake rationalization" is a bit backwards. We say that many actions happen before we are aware of it or have a rationalization for them but becoming aware of why we did something after the fact doesn't mean that "you" weren't part of the decision. Aren't your reflexes a part of "you"?
- bobcostas55 10y ago
  Of course, but nobody is going to say "my reflexes did it". We come up with stories, lines of argumentation, etc. Even though these are "ex post" we often manage to convince ourselves... In Homer's time the Greeks viewed the unconscious as an external (divine) force, the most famous example being Telemachus's sneeze. In a weird way this feels more intellectually honest than the thing we do.
  - peteretep 10y ago
    > we often manage to convince ourselves In this case, presumably the trick is to convince a jury.

wavefunction 10y ago

I don't see how this is true. With a machine system you would at least have the ability to log the mathematical operations and results, which would be complex, sure. But more complex than figuring out if another person is lying about a very nuanced and opaque decision they made?

feel_the_need 10y ago
The scope of your question is too large. Do not compare the complexity of AI to the human brain. Instead compare AI to traditional robotics. In expert systems rules are laid out and logic chains can be followed. AI is a black box of a billion random weighted nodes. The system does not care about 'why' when reducing loss.

catwell 10y ago

Yes. This is why https://news.ycombinator.com/item?id=13111516 https://news.ycombinator.com/item?id=13111516 was on the front page last month.

javiramos 10y ago

NuTonomy has an autonomous driving technology based on formal logic [1]. It seems like formal logic is a better approach for retroactively explaining software decisions. [1] http://spectrum.ieee.org/transportation/self-driving/after-mastering-singapores-streets-nutonomys-robotaxis-are-poised-to-take-on-new-cities http://spectrum.ieee.org/transportation/self-driving/after-m...

Fricken 10y ago
Google too uses plenty of formal logic in their autonomous OS, it's a frankenstein of various machine learning techniques. Narrow pattern recognition alone, while powerful, is unlikely to ever be suitable for things like negotiating busy four way stops. There is the presumption that all you gotta do is gather enough training data and shazam!, you got a self driving car, but that's bollocks. There is a tremendous amount of elbow grease that foes into developing and validating an AV to six sigma reliability. Six sigma might not even be good enough for something as safety-critical as a driverless car.
- rthille 10y ago
  It doesn't need six-sigma, just to be significantly better than humans, and for the manufacturer to take on the liability.
  - Fricken 10y ago
    In strictly rational terms It has to be better than 1 fatality for every 100,000,000 miles driven, but public opinion isn't exactly rational so it probably needs to be even better than that.
    - xorxornop 10y ago
      It'll be that way at first, yeah, but once it becomes obvious how much better the AI drivers are, it will start to become irresponsible to not use it

whistlerbrk 10y ago

I feel like this is very actively being addressed, for example by visualization of intermediate features.