6 ms·

I have been yelling about this for a long time: Tesla is not going to be able to deliver full self-driving as promised. They don't have the hardware, for one (r

by tbabb 8y ago

I have been yelling about this for a long time: Tesla is not going to be able to deliver full self-driving as promised. They don't have the hardware, for one (ranging is terrible; they need stereo cameras), and second, their software strategy seems to be a dead end. They need sensor fusion. The system needs to make maximum use of all the information available to it: Where is the road striping? Where are the other cars going? Where are the road signs and signals? (If there's one in your path, you certainly shouldn't drive into it!) Are there camera-visible obstructions? What were the interpretations and actions of previous Tesla trips along the same route? In these problem cases, all data except the left and right lane striping seems to be completely ignored. There was even more information at the fatal offramp location (cross-striping over the lane separation zone), which the vehicle drove straight over. The system is not making maximum use of the information available to it, in fact it is using hardly any of it at all, and fixating on what it thinks is a single most salient piece of data. Sensor fusion algorithms tend to behave the opposite way-- each additional piece of data informs the interpretation of all the other data. You can have very poor-quality data, but if it is even moderately over-constrained, your state estimate can be very good in spite of it. I think it would be completely reasonable to have a neural net in the loop of a sensor fusion algorithm, with fusion constraints informing the NN's interpretation, and the NN's estimates feeding back into the fusion algorithm as uncertain data. IMO Tesla will do at least one of: * Very expensively retract their promise of full self driving for delivered vehicles * Completely overhaul/redesign their driving software and start again nearly from scratch * Get into a regulatory/legal tangle with the NTSA/courts/DOJ over all the dead people their system is making.

Dumblydorr 8y ago
Where is your evidence for these claims?
eclipxe 8y ago
They do have stereo cameras and sensor fusing, as well as detecting more than just the lines in the road. Here is what the camera sees: https://www.youtube.com/watch?v=rACZACXgreQ https://www.youtube.com/watch?v=rACZACXgreQ
- tbabb 8y ago
  What in that video suggests sensor and/or stereo fusion to you? I notice that the temporal coherence is pretty bad-- Pedestrians pop out of recognition when they go behind trees; lane/exit boundaries wiggle all over the place and occasionally frame-pop into different configurations. A kalman filter, for example, is a state estimator which maintains temporal coherence, and makes heavy use of previous estimates/sensor inference when computing the most updated estimate. It doesn't look to me like that kind of strategy is being used to maintain the vehicle's world model. IMO a good estimator wouldn't treat "a pedestrian popping out of existence" as the most likely estimate for any circumstance, let alone one where they were clearly present in the previous 50 frames. I don't doubt they're using KF on the vehicle's inertial movement, but based on the failures and this video, it sure doesn't look like it's using a fusion technique for the world model. There are left and right-looking cameras, but the FOV overlap between them is not very substantial, and there can't be stereopsis where there is no overlap. Per the Tesla website, there are three forward-looking cameras, and they each have a different FOV. The parallax baseline between them is only a few centimeters, too, so the depth sensitivity isn't going to be spectacular. It's certainly possible that there could be some narrow-baseline stereo fusion, but it could only really happen inside the narrowest field of view, where the coverage overlaps with more than one camera. That's the circumstance where having a narrow baseline would hurt the most. Based on that it doesn't really seem like the system is well set-up for stereopsis; if it's there it seems like an afterthought. I could certainly be wrong, as I don't have access to the code. Are you going by some other secondary source/information?
  - ben174 8y ago
    Certainly two cameras with different FOVs could be combined to give the same depth data that a stereo camera setup could give, right?
    - dmitrygr 8y ago
      not if they are on the same axis. then anything along that axis cannot have its depth determined
tim333 8y ago
Musk is more optimistic than you - says late 2020 for sleep while the car drives https://www.youtube.com/watch?v=Y8dEYm8hzLo&t=10m25s https://www.youtube.com/watch?v=Y8dEYm8hzLo&t=10m25s Also quite interestingly he says there will be a big jump forward in quality when they switch to their own computing hardware (18m40 or so)
- FireBeyond 8y ago
  He's also said 2016. And 2018.
  - quickthrower2 8y ago
    He likes even numbers.
    - x38iq84n 8y ago
      2048?
- x38iq84n 8y ago
  > late 2020 for sleep while the car drives Musk is often overly optimistic and he keeps underestimating the problem at hand. I call BS on this, it won't be ready in 5 or even 10 years. And then there is a regulatory approval.
  - tim333 8y ago
    Indeed though he's delivered some stuff. It'll be interesting to see how it goes.
    - tim333 8y ago
      Though the 2017 LA to NYC drive is behind schedule. (https://www.theverge.com/2016/10/19/13341100/tesla-self-driving-autonomous-road-trip-la-nyc https://www.theverge.com/2016/10/19/13341100/tesla-self-driv...)
- 8y ago
ec109685 8y ago
I think you are confusing what the software is capable of today with hardware limitations in the sensors deployed in the cars.

by tbabb 8y ago

Dumblydorr 8y ago

Where is your evidence for these claims?

eclipxe 8y ago

They do have stereo cameras and sensor fusing, as well as detecting more than just the lines in the road. Here is what the camera sees: https://www.youtube.com/watch?v=rACZACXgreQ https://www.youtube.com/watch?v=rACZACXgreQ

tbabb 8y ago
What in that video suggests sensor and/or stereo fusion to you? I notice that the temporal coherence is pretty bad-- Pedestrians pop out of recognition when they go behind trees; lane/exit boundaries wiggle all over the place and occasionally frame-pop into different configurations. A kalman filter, for example, is a state estimator which maintains temporal coherence, and makes heavy use of previous estimates/sensor inference when computing the most updated estimate. It doesn't look to me like that kind of strategy is being used to maintain the vehicle's world model. IMO a good estimator wouldn't treat "a pedestrian popping out of existence" as the most likely estimate for any circumstance, let alone one where they were clearly present in the previous 50 frames. I don't doubt they're using KF on the vehicle's inertial movement, but based on the failures and this video, it sure doesn't look like it's using a fusion technique for the world model. There are left and right-looking cameras, but the FOV overlap between them is not very substantial, and there can't be stereopsis where there is no overlap. Per the Tesla website, there are three forward-looking cameras, and they each have a different FOV. The parallax baseline between them is only a few centimeters, too, so the depth sensitivity isn't going to be spectacular. It's certainly possible that there could be some narrow-baseline stereo fusion, but it could only really happen inside the narrowest field of view, where the coverage overlaps with more than one camera. That's the circumstance where having a narrow baseline would hurt the most. Based on that it doesn't really seem like the system is well set-up for stereopsis; if it's there it seems like an afterthought. I could certainly be wrong, as I don't have access to the code. Are you going by some other secondary source/information?
- ben174 8y ago
  Certainly two cameras with different FOVs could be combined to give the same depth data that a stereo camera setup could give, right?
  - dmitrygr 8y ago
    not if they are on the same axis. then anything along that axis cannot have its depth determined

tim333 8y ago

Musk is more optimistic than you - says late 2020 for sleep while the car drives https://www.youtube.com/watch?v=Y8dEYm8hzLo&t=10m25s https://www.youtube.com/watch?v=Y8dEYm8hzLo&t=10m25s Also quite interestingly he says there will be a big jump forward in quality when they switch to their own computing hardware (18m40 or so)

FireBeyond 8y ago
He's also said 2016. And 2018.
- quickthrower2 8y ago
  He likes even numbers.
  - x38iq84n 8y ago
    2048?
x38iq84n 8y ago
> late 2020 for sleep while the car drives Musk is often overly optimistic and he keeps underestimating the problem at hand. I call BS on this, it won't be ready in 5 or even 10 years. And then there is a regulatory approval.
- tim333 8y ago
  Indeed though he's delivered some stuff. It'll be interesting to see how it goes.
  - tim333 8y ago
    Though the 2017 LA to NYC drive is behind schedule. (https://www.theverge.com/2016/10/19/13341100/tesla-self-driving-autonomous-road-trip-la-nyc https://www.theverge.com/2016/10/19/13341100/tesla-self-driv...)
8y ago

ec109685 8y ago

I think you are confusing what the software is capable of today with hardware limitations in the sensors deployed in the cars.