3 ms·

> If they had the ability to measure confidence in the system, they would use it to issue warnings or disable autopilot Yeah, that's basically what I'm asking

> If they had the ability to measure confidence in the system, they would use it to issue warnings or disable autopilot Yeah, that's basically what I'm asking for, maybe a warning light at a certain threshold would be a better default, personally I'd still find a number more trustworthy though. > The "confidence meter" would almost certainly be 100% right up until it crashes into an obvious obstacle If image recognition algorithms have associated confidence levels, I'd be surprised something more complicated like road navigation was 100% certain all the time.

simias 9y ago
>I'd be surprised something more complicated like road navigation was 100% certain all the time. If for instance it failed to detect an obstacle how could it tell you that it failed detecting an obstacle? Or that it got the road markings wrong? Given that the consequences for making a mistake might be people dying I certainly hope that's not how it works under the hood. Anything less than "I'm absolutely sure that I'm not currently speeding towards an obstacle" should automatically trigger a safety system that would cause the autopilot to disengage and give the control back to the driver, maybe engaging the emergency lights to let the other drivers know that something is amiss. I wouldn't be surprised if in the moments preceding these crashes the autopilot algorithm was completely sure it was doing the right thing. Of course these various crashes show that in fact the algorithm makes mistakes sometimes but I certainly hope that it's not aware that it's making mistakes, it just messes up while it thinks everything is going as planned.
- Piskvorrr 9y ago
  Anything less than "I'm absolutely sure that I'm not currently speeding towards an obstacle" - that's an unattainable goal; there's no way to actually reach this, whether a human or a robot is driving. I'm overwhelmingly sure during the most part of my drives, and mostly sure for the rest - but absolutely? Never.
salawat 9y ago
The problem is, given a neural network, the decided course of action IS at 100% confidence. Unless you design the network to specifically answer "how similar is this?" and pair it with a training set and the output of another executing neural net, a "confidence meter" isn't possible. This is one of the trickier bits to wrap one's mind around with neural networks. The systems we train for specialized tasks have no concept of 'confidence' or being wrong. There is no meta-awareness that judges how well a task is being performed in real-time outside of the training environment. Humans don't suffer from this issue (as much) due to the mind bogglingly large and complex neural nets we have to deal with the world. Every time you set yourself to practicing a task, you are putting a previously trained neural network through further training and evolution. You can recognize when you are doing it because the process of doing so takes conscious effort. You are not 'just doing <the task>', you are 'doing <the task>, and comparing your performance against an ideation or metric of how you WANT <the task> to be performed'. This process is basically your pre-frontal cortex and occipital lobe for instance, tuning your hippocampus, sensory and motor cortex to perfect the perfect tennis swing. When we train visual neural networks, we're talking levels of intelligence supplied by the occipital lobe , hippocampus, alone. Imagine every time you hear about a neural net that a human was lobotomized until they could perform ONLY that task with any reliability. Kinda changes the comfort level with letting one of these take the wheel doesn't it? Neural nets are REALLY neat. Don't get me wrong. I love them. Unfortunately, the real world is also INCREDIBLY hard to safely operate in. Many little things that humans 'just do' are the results of our own brains 'hacking' together older neural nets with a higher level one.
- derangedHorse 9y ago
  There is actually some sort of classification confidence regarding outputs. It's not a binary yes or no for output neurons regardless of what abstraction you've observed in whichever tensorflow or other high level API being used.
- Eruditass 9y ago
  Have you actually worked on applying neural networks in the real world? Calibrating the probabilities of machine learning algorithms is an old problem. By nature of disciminative algorithms and increasing model capacities, yes training typically pushes outputs to one extreme. There is a ton of information still maintained which can be properly calibrated for downstream ingestion, which anyone actually trying to integrate these into actual applications should be doing. Most recently: https://arxiv.org/abs/1706.04599 https://arxiv.org/abs/1706.04599 https://arxiv.org/abs/1802.03916 https://arxiv.org/abs/1802.03916

simias 9y ago

>I'd be surprised something more complicated like road navigation was 100% certain all the time. If for instance it failed to detect an obstacle how could it tell you that it failed detecting an obstacle? Or that it got the road markings wrong? Given that the consequences for making a mistake might be people dying I certainly hope that's not how it works under the hood. Anything less than "I'm absolutely sure that I'm not currently speeding towards an obstacle" should automatically trigger a safety system that would cause the autopilot to disengage and give the control back to the driver, maybe engaging the emergency lights to let the other drivers know that something is amiss. I wouldn't be surprised if in the moments preceding these crashes the autopilot algorithm was completely sure it was doing the right thing. Of course these various crashes show that in fact the algorithm makes mistakes sometimes but I certainly hope that it's not aware that it's making mistakes, it just messes up while it thinks everything is going as planned.

Piskvorrr 9y ago
Anything less than "I'm absolutely sure that I'm not currently speeding towards an obstacle" - that's an unattainable goal; there's no way to actually reach this, whether a human or a robot is driving. I'm overwhelmingly sure during the most part of my drives, and mostly sure for the rest - but absolutely? Never.

salawat 9y ago

The problem is, given a neural network, the decided course of action IS at 100% confidence. Unless you design the network to specifically answer "how similar is this?" and pair it with a training set and the output of another executing neural net, a "confidence meter" isn't possible. This is one of the trickier bits to wrap one's mind around with neural networks. The systems we train for specialized tasks have no concept of 'confidence' or being wrong. There is no meta-awareness that judges how well a task is being performed in real-time outside of the training environment. Humans don't suffer from this issue (as much) due to the mind bogglingly large and complex neural nets we have to deal with the world. Every time you set yourself to practicing a task, you are putting a previously trained neural network through further training and evolution. You can recognize when you are doing it because the process of doing so takes conscious effort. You are not 'just doing <the task>', you are 'doing <the task>, and comparing your performance against an ideation or metric of how you WANT <the task> to be performed'. This process is basically your pre-frontal cortex and occipital lobe for instance, tuning your hippocampus, sensory and motor cortex to perfect the perfect tennis swing. When we train visual neural networks, we're talking levels of intelligence supplied by the occipital lobe , hippocampus, alone. Imagine every time you hear about a neural net that a human was lobotomized until they could perform ONLY that task with any reliability. Kinda changes the comfort level with letting one of these take the wheel doesn't it? Neural nets are REALLY neat. Don't get me wrong. I love them. Unfortunately, the real world is also INCREDIBLY hard to safely operate in. Many little things that humans 'just do' are the results of our own brains 'hacking' together older neural nets with a higher level one.

derangedHorse 9y ago
There is actually some sort of classification confidence regarding outputs. It's not a binary yes or no for output neurons regardless of what abstraction you've observed in whichever tensorflow or other high level API being used.
Eruditass 9y ago
Have you actually worked on applying neural networks in the real world? Calibrating the probabilities of machine learning algorithms is an old problem. By nature of disciminative algorithms and increasing model capacities, yes training typically pushes outputs to one extreme. There is a ton of information still maintained which can be properly calibrated for downstream ingestion, which anyone actually trying to integrate these into actual applications should be doing. Most recently: https://arxiv.org/abs/1706.04599 https://arxiv.org/abs/1706.04599 https://arxiv.org/abs/1802.03916 https://arxiv.org/abs/1802.03916