4 ms·

The point is that they can explain why they are taking a left turn!

by debdut 5y ago

jameshart 5y ago
Can they? How exactly does your brain do route planning? How do you pick which lane to turn down in the parking lot when looking for a space? Why did you get off at exit 14? Wait, this isn’t even your turn, why did you go left here? I mean, when you get down to it, why are you even driving to this deadend job? But yeah, sure, humans can ‘explain’ their behavior, which is why we can trust them.
- neatze 5y ago
  Well humans can explain critical aspect of what they doing while teaching other humans, furthermore they can improve explaining while teaching to be more effective in real time in current context, such as using analogies, stories, etc.
- YeGoblynQueenne 5y ago
  >> How exactly does your brain do route planning? That is not the kind of explanation that is needed in AI systems. When people talk about "explainable AI", they literally just mean systems that can answer the kind of question that a human would be able to answer. That's because a question that a human cannot answer is very likely to have an answer that a human will either not be able to understand, or will have to work very hard to understand... which is no better than no explanation
  - jameshart 5y ago
    I think what people are looking for in ‘explainable AI’ is: when the AI makes a bad decision, they want to be able to look into the neural network and say ‘there: that neuron being set to that value is what made the AI mistake the cyclist for a drop kerb’. Then we can fix the value and the AI will not make that mistake again. But when an AI gets sufficiently complex of course there won’t be explanations that make sense for those kinds of errors, because just like a human the AI is integrating lots of different bits of information that it has learned are important and it has limited capacity and sometimes it just gets its attention focused on the wrong thing and it just didn’t see the guy, okay? Demanding that AI be explainable is fundamentally demanding that it not be intelligent.
    - YeGoblynQueenne 5y ago
      Just to be clear, a neural network is not "an AI". "AI" is the name of the research field. We don't have "AIs" as in Science Fiction yet, and neural networks don't do anything "just like a human". When people in AI research talk about "attention" in neural networks, that's just an anthropomorphic, and quite unfortunate, name for a specific technique in training neural nets. It doesn't mean that a machine vision system has the ability to focus its attention in the same way that humans do. That out of the way, there are AI approaches that can explain their actions just fine without going dumb. For example, I posted this comment earlier: https://news.ycombinator.com/item?id=30872400 https://news.ycombinator.com/item?id=30872400 about an AI system called Nook that recently won a tournament against 8 human champions of the card game Bridge. In Bridge, players must be able to explain their moves, so an AI player without the ability to explain its decisions can't play a full game of Bridge.

The point is that they can explain why they are taking a left turn!

by debdut 5y ago

The point is that they can explain why they are taking a left turn!

Can they? How exactly does your brain do route planning? How do you pick which lane to turn down in the parking lot when looking for a space? Why did you get off at exit 14? Wait, this isn’t even your turn, why did you go left here? I mean, when you get down to it, why are you even driving to this deadend job? But yeah, sure, humans can ‘explain’ their behavior, which is why we can trust them.

neatze 5y ago
Well humans can explain critical aspect of what they doing while teaching other humans, furthermore they can improve explaining while teaching to be more effective in real time in current context, such as using analogies, stories, etc.
YeGoblynQueenne 5y ago
>> How exactly does your brain do route planning? That is not the kind of explanation that is needed in AI systems. When people talk about "explainable AI", they literally just mean systems that can answer the kind of question that a human would be able to answer. That's because a question that a human cannot answer is very likely to have an answer that a human will either not be able to understand, or will have to work very hard to understand... which is no better than no explanation
- jameshart 5y ago
  I think what people are looking for in ‘explainable AI’ is: when the AI makes a bad decision, they want to be able to look into the neural network and say ‘there: that neuron being set to that value is what made the AI mistake the cyclist for a drop kerb’. Then we can fix the value and the AI will not make that mistake again. But when an AI gets sufficiently complex of course there won’t be explanations that make sense for those kinds of errors, because just like a human the AI is integrating lots of different bits of information that it has learned are important and it has limited capacity and sometimes it just gets its attention focused on the wrong thing and it just didn’t see the guy, okay? Demanding that AI be explainable is fundamentally demanding that it not be intelligent.
  - YeGoblynQueenne 5y ago
    Just to be clear, a neural network is not "an AI". "AI" is the name of the research field. We don't have "AIs" as in Science Fiction yet, and neural networks don't do anything "just like a human". When people in AI research talk about "attention" in neural networks, that's just an anthropomorphic, and quite unfortunate, name for a specific technique in training neural nets. It doesn't mean that a machine vision system has the ability to focus its attention in the same way that humans do. That out of the way, there are AI approaches that can explain their actions just fine without going dumb. For example, I posted this comment earlier: https://news.ycombinator.com/item?id=30872400 https://news.ycombinator.com/item?id=30872400 about an AI system called Nook that recently won a tournament against 8 human champions of the card game Bridge. In Bridge, players must be able to explain their moves, so an AI player without the ability to explain its decisions can't play a full game of Bridge.