5 ms·

What were his reasons? Searching the web reveals lots of machismo and assorted hero worship, but no actual solid technical arguments. Nobody in the field seems

What were his reasons? Searching the web reveals lots of machismo and assorted hero worship, but no actual solid technical arguments. Nobody in the field seems to think a vision-only solution is practical, other than Tesla, who are also not providing solid technical arguments that I can find. This IEEE item seems to summarize the situation well. https://spectrum.ieee.org/tesla-places-big-bet-vision-only-self-driving https://spectrum.ieee.org/tesla-places-big-bet-vision-only-s...

LegitShady 5y ago
It was a discussion about how such a system must function with all the different inputs vs the vision only model. If you have radar and lidar and vision, then you have at least three different specialist machine learning models running, and then another model running that takes their outputs and decides what the car is going to do. You may have even more than that, some doing specific tasks like localization. Neural nets and vision only is a more difficult but in the long run straightforward solution. The example he brought up was alphazero vs some other chess engine that has a rook engine, and a knight engine, etc. Basically he's backing the end to end neural network back approach over some kind of multi-sensor fusion. https://youtu.be/_L3gNaAVjQ4?t=3801 https://youtu.be/_L3gNaAVjQ4?t=3801
- YZF 5y ago
  That argument doesn't make any sense though. Clearly radar + lidar + vision is a superset of vision only. It can perform as well as vision only if you disable the other two. So the claim is there is absolutely no scenario where the other systems can contribute which seems to be false. E.g. Tesla's cameras are blinded pointing straight on into the sun, the car even tells you the cameras are blinded. If the cameras are splashed with mud they'll also see nothing. Tesla's radar was able to see "through" some obstacles which the vision system can not (and you can see that in the traffic visualizations). Now how far do you want to fuse, do you just want to overlap the unique sensing abilities of each of those systems? How do you handle conflict when you're in the regions where all systems sense "equally" well. Sure, there's questions. But clearly having more data can't make things worse. Elon's argument that that's the way to go because that's how humans drive is just ridiculous. And I say this as a proud Model 3 owner (great car, will never ever be autonomous, I don't care). It doesn't pass the sniff test.
  - jfoster 5y ago
    The real equation is closer to: Radar + LIDAR + Vision + Cost + Latency - Battery Doing it with vision only saves on cost, latency & energy consumption.
    - YZF 5y ago
      I agree it saves on cost and energy consumption. Not sure about latency. But it's an inferior sensing system [and] it remains to be seen whether it can do the job or not, so far seems like not. Even if it can do the job (by some measure) whether it can outperform the better sensing systems which seems unlikely. I'd pay a little more for better safety, we know the cost of "human safety" as it reflects in our insurance premiums...
      - jfoster 5y ago
        If fusing two data sources is necessary prior to making actuation decisions, how could adding another data source not introduce additional processing & latency? It's not just about paying more dollars, but also range. You're assuming that a dual system will be safer, but what if such systems are more prone to perception confusion, or other anomalies? Even if a dual system were safer, it doesn't make sense to say that you will pay for safety absolutely. For example, you can always add an additional smoke alarm to your house for some marginal improvement in safety, but at some point most people decide they are safe enough.
  - olau 5y ago
    > So the claim is there is absolutely no scenario where the other systems can contribute which seems to be false. No. The claim was that pouring resources into those other systems is better spent improving the vision system, within the context that Tesla and Comma.ai is operating. > But clearly having more data can't make things worse. There are several examples of cars hitting stuff because one part of the sensor suite thought there was a problem, or that there wasn't a problem. In general, the more complex you make a system, the more complex the failure modes get.
kanwisher 5y ago
Hotz makes his own company making self driving using vision only https://comma.ai/ https://comma.ai/

LegitShady 5y ago

It was a discussion about how such a system must function with all the different inputs vs the vision only model. If you have radar and lidar and vision, then you have at least three different specialist machine learning models running, and then another model running that takes their outputs and decides what the car is going to do. You may have even more than that, some doing specific tasks like localization. Neural nets and vision only is a more difficult but in the long run straightforward solution. The example he brought up was alphazero vs some other chess engine that has a rook engine, and a knight engine, etc. Basically he's backing the end to end neural network back approach over some kind of multi-sensor fusion. https://youtu.be/_L3gNaAVjQ4?t=3801 https://youtu.be/_L3gNaAVjQ4?t=3801

YZF 5y ago
That argument doesn't make any sense though. Clearly radar + lidar + vision is a superset of vision only. It can perform as well as vision only if you disable the other two. So the claim is there is absolutely no scenario where the other systems can contribute which seems to be false. E.g. Tesla's cameras are blinded pointing straight on into the sun, the car even tells you the cameras are blinded. If the cameras are splashed with mud they'll also see nothing. Tesla's radar was able to see "through" some obstacles which the vision system can not (and you can see that in the traffic visualizations). Now how far do you want to fuse, do you just want to overlap the unique sensing abilities of each of those systems? How do you handle conflict when you're in the regions where all systems sense "equally" well. Sure, there's questions. But clearly having more data can't make things worse. Elon's argument that that's the way to go because that's how humans drive is just ridiculous. And I say this as a proud Model 3 owner (great car, will never ever be autonomous, I don't care). It doesn't pass the sniff test.
- jfoster 5y ago
  The real equation is closer to: Radar + LIDAR + Vision + Cost + Latency - Battery Doing it with vision only saves on cost, latency & energy consumption.
  - YZF 5y ago
    I agree it saves on cost and energy consumption. Not sure about latency. But it's an inferior sensing system [and] it remains to be seen whether it can do the job or not, so far seems like not. Even if it can do the job (by some measure) whether it can outperform the better sensing systems which seems unlikely. I'd pay a little more for better safety, we know the cost of "human safety" as it reflects in our insurance premiums...
    - jfoster 5y ago
      If fusing two data sources is necessary prior to making actuation decisions, how could adding another data source not introduce additional processing & latency? It's not just about paying more dollars, but also range. You're assuming that a dual system will be safer, but what if such systems are more prone to perception confusion, or other anomalies? Even if a dual system were safer, it doesn't make sense to say that you will pay for safety absolutely. For example, you can always add an additional smoke alarm to your house for some marginal improvement in safety, but at some point most people decide they are safe enough.
- olau 5y ago
  > So the claim is there is absolutely no scenario where the other systems can contribute which seems to be false. No. The claim was that pouring resources into those other systems is better spent improving the vision system, within the context that Tesla and Comma.ai is operating. > But clearly having more data can't make things worse. There are several examples of cars hitting stuff because one part of the sensor suite thought there was a problem, or that there wasn't a problem. In general, the more complex you make a system, the more complex the failure modes get.

kanwisher 5y ago

Hotz makes his own company making self driving using vision only https://comma.ai/ https://comma.ai/