3 ms·

this almost throwaway line at the end caught my eye: "The safest bet, then, is for automakers to use an array of sensors, in order to build redundancy into thei

this almost throwaway line at the end caught my eye: "The safest bet, then, is for automakers to use an array of sensors, in order to build redundancy into their systems." experienced engineers know that redundancy is a double-edged sword. what do you do when the data from redundant systems disagree? having more data sources means dealing with more edge cases. a single reliable data source (as in the limitations and biases are well known) is better than two less predictable ones. i'm not saying that that's necessarily the case here, but it's not as obvious as this line suggests that the answer is just to add more kinds of sensors (and thereby add more diverse data sources).

catbird 9y ago
I disagree. In an autonomous system, more sensors is almost always better, provided you have enough computing power to process the data. Having two sensors that disagree gives you much more information than if you had a single sensor with a malfunction. In the first case, you know there's some kind of fault and can take steps to fix it. In the second, you're running off bad data and who knows what could happen. Three sensors are better than two—it's a common setup in flight control systems to have three identical computers running all calculations, and the majority opinion is taken as truth.
- clairity 9y ago
  yes, in a resource-unconstrained world, we could put every sensor on the car and add as much computing power as we'd like. in your 2-sensor scenario, how do you know which one is providing incorrect data?
unityByFreedom 9y ago
> what do you do when the data from redundant systems disagree? Self driving car systems' rules aren't directly decided by humans. The system is trained to come up with its own rules given sensor input. So ideally, you have more different types of input from diverse sources. Machine learning is a lot easier when you have a diverse dataset. > having more data sources means dealing with more edge cases. Yes, and that's a good thing. You want the system to be able to handle the edge case from the article, and a diverse array of sensors can do that. > a single reliable data source (as in the limitations and biases are well known) is better than two less predictable ones. Can you provide an example of a single vehicle sensor that would be preferable to two others? I can't see why this decision would be made, aside from attempts to lower cost. Two sensors provides more reliability than one.
- clairity 9y ago
  the idea that more is always better is the common misperception that i'm addressing. if you've ever tried to design a system with multiple, sometimes conflicting inputs, complexity is a real cost to think about (not to mention financial cost). two sensors are often in a master-slave relationship to avoid conflicts, and as catbird mentioned, if you need peering redundancy, then three is much better than two (but that's not perfect either), so that there's essentially error correction built in. machine learning is cool, but it's not a panacea. there are plenty of meticulously trained AI's that fail comically in novel situations (like the example in the original article). afaict, we still don't have a lot of knowledge around the boundary conditions of the machine learning systems themselves.
  - unityByFreedom 9y ago
    > the idea that more is always better is the common misperception that i'm addressing It's not a misperception in the case of machine learning. The system can easily discard data that isn't useful at a given point in time. When it is determined to be useful, it's great to have it there. I'd suggest trying out some machine learning yourself to get a good understanding of why more data is better. Tutorials for Kaggle's Titanic competition can be a good primer. > if you've ever tried to design a system with multiple, sometimes conflicting inputs, complexity is a real cost to think about (not to mention financial cost). Not when the system is using machine learning, particularly neural networks. All the self driving car systems are using deep convolutional NNs, so the complexity of more data is helpful, not a hindrance. > machine learning is cool, but it's not a panacea. there are plenty of meticulously trained AI's that fail comically in novel situations (like the example in the original article) The failure in the article was due to lack of diverse sensor input, not because the system was built using machine learning. You're right that machine learning isn't a panacea. The limits aren't well defined. Still, in the case of self driving cars, deep learning is state of the art. Nobody is hand-coding rules for how to drive the car. It's all about having as much good data as you can gather.
tormeh 9y ago
>what do you do when the data from redundant systems disagree That's a luxury problem. It's like saying that you're better off without ECC because you don't have to deal with memory corruption. With a single sensor, you still have to deal with inaccuracies, the difference is that with only one you just don't know if you have it or not.
- clairity 9y ago
  the luxury is adding every kind of sensor without considering cost, engineering time, computing power, etc. complexity is extra-linear, so you have to have to do the cost-benefit analysis to determine if it's worth it. and it may well be, but my point was that it's not a knee-jerk answer to always add more sensors. there is clear benefit, but is it worth the cost?
yellowapple 9y ago
Except in this case there's no "single reliable data source". All of the useful data sources are limited in some way. You need a diverse array of data sources to fill in the gaps of each individual source. Not to mention that one should always strive to avoid a single point of failure. A "single reliable data source" is - whaddya know - a single point of failure. If that data source stops being "reliable", then you're screwed. When you're screwed in a giant hunk of metal rolling down a highway at 65+ miles per hour, then the only system you have left is to pray to your god(s) for a good afterlife.

catbird 9y ago

I disagree. In an autonomous system, more sensors is almost always better, provided you have enough computing power to process the data. Having two sensors that disagree gives you much more information than if you had a single sensor with a malfunction. In the first case, you know there's some kind of fault and can take steps to fix it. In the second, you're running off bad data and who knows what could happen. Three sensors are better than two—it's a common setup in flight control systems to have three identical computers running all calculations, and the majority opinion is taken as truth.

clairity 9y ago
yes, in a resource-unconstrained world, we could put every sensor on the car and add as much computing power as we'd like. in your 2-sensor scenario, how do you know which one is providing incorrect data?

unityByFreedom 9y ago

> what do you do when the data from redundant systems disagree? Self driving car systems' rules aren't directly decided by humans. The system is trained to come up with its own rules given sensor input. So ideally, you have more different types of input from diverse sources. Machine learning is a lot easier when you have a diverse dataset. > having more data sources means dealing with more edge cases. Yes, and that's a good thing. You want the system to be able to handle the edge case from the article, and a diverse array of sensors can do that. > a single reliable data source (as in the limitations and biases are well known) is better than two less predictable ones. Can you provide an example of a single vehicle sensor that would be preferable to two others? I can't see why this decision would be made, aside from attempts to lower cost. Two sensors provides more reliability than one.

clairity 9y ago
the idea that more is always better is the common misperception that i'm addressing. if you've ever tried to design a system with multiple, sometimes conflicting inputs, complexity is a real cost to think about (not to mention financial cost). two sensors are often in a master-slave relationship to avoid conflicts, and as catbird mentioned, if you need peering redundancy, then three is much better than two (but that's not perfect either), so that there's essentially error correction built in. machine learning is cool, but it's not a panacea. there are plenty of meticulously trained AI's that fail comically in novel situations (like the example in the original article). afaict, we still don't have a lot of knowledge around the boundary conditions of the machine learning systems themselves.
- unityByFreedom 9y ago
  > the idea that more is always better is the common misperception that i'm addressing It's not a misperception in the case of machine learning. The system can easily discard data that isn't useful at a given point in time. When it is determined to be useful, it's great to have it there. I'd suggest trying out some machine learning yourself to get a good understanding of why more data is better. Tutorials for Kaggle's Titanic competition can be a good primer. > if you've ever tried to design a system with multiple, sometimes conflicting inputs, complexity is a real cost to think about (not to mention financial cost). Not when the system is using machine learning, particularly neural networks. All the self driving car systems are using deep convolutional NNs, so the complexity of more data is helpful, not a hindrance. > machine learning is cool, but it's not a panacea. there are plenty of meticulously trained AI's that fail comically in novel situations (like the example in the original article) The failure in the article was due to lack of diverse sensor input, not because the system was built using machine learning. You're right that machine learning isn't a panacea. The limits aren't well defined. Still, in the case of self driving cars, deep learning is state of the art. Nobody is hand-coding rules for how to drive the car. It's all about having as much good data as you can gather.

tormeh 9y ago

>what do you do when the data from redundant systems disagree That's a luxury problem. It's like saying that you're better off without ECC because you don't have to deal with memory corruption. With a single sensor, you still have to deal with inaccuracies, the difference is that with only one you just don't know if you have it or not.

clairity 9y ago
the luxury is adding every kind of sensor without considering cost, engineering time, computing power, etc. complexity is extra-linear, so you have to have to do the cost-benefit analysis to determine if it's worth it. and it may well be, but my point was that it's not a knee-jerk answer to always add more sensors. there is clear benefit, but is it worth the cost?

yellowapple 9y ago

Except in this case there's no "single reliable data source". All of the useful data sources are limited in some way. You need a diverse array of data sources to fill in the gaps of each individual source. Not to mention that one should always strive to avoid a single point of failure. A "single reliable data source" is - whaddya know - a single point of failure. If that data source stops being "reliable", then you're screwed. When you're screwed in a giant hunk of metal rolling down a highway at 65+ miles per hour, then the only system you have left is to pray to your god(s) for a good afterlife.