3 ms·

I wonder if Tesla has a way to visualize how disruptive a change to Autopilot. They've supposedly got all this driving data lying around. Would they be able to

I wonder if Tesla has a way to visualize how disruptive a change to Autopilot. They've supposedly got all this driving data lying around. Would they be able to apply the updated algorithms to that data and quantify how many decisions would be affected, and by how much? i.e. 30% of past decisions would have been changed with the new algorithm, 5% changed by an angle of over 2 degrees. On a similar note, shouldn't customers be getting release notes for these updates? I let my customers know when we're updating a web page. Tesla should surely be informing them when cars are going to start lane centering.

donkeyd 9y ago
How do you define a 'decision' in something like autopilot though? Also, if I remember correctly, only a couple of images per second are sent to Tesla. If there's a fraction of a second where it may decide to run into something, it may not actually appear in test results.
- maxerickson 9y ago
  One way is to extract the drive path for some given sensor input. Then you can classify the drive path as safe, marginal, collision, severe collision, etc. Changes from safe to safe aren't interesting. Changes in the wrong direction are obvious issues.
walrus01 9y ago
I really don't think they could do this effectively unless each car also uploads 20GB per day of 60fps high quality video, which would collapse the cellular networks they use.
makomk 9y ago
I don't think they have that much data that's useful for this purpose. They may have billions of miles of some kinds of data, but they only upload occasional samples of the camera video that would be required to test a feature of this sort. I can't find the exact link now, but someone reverse-engineered it and it was something on the order of a few seconds a few times an hour.
kalleboo 9y ago
The problem is you can't simulate the video that happens after your new decision. So this steering update would take you 0.3 degrees to the right. So what? Well those 0.3 degrees might change the angle of the line which influences the car to steer another 0.3 degrees, etc. But without that followup video (which you can't simulate from a 2D recording) you don't know how it would react to the change in environment. The only way to regression test these things is in a simulated 3D environment (or miles and miles of real test track)
- darkerside 9y ago
  Following the branch down is obviously impossible, but simply detecting that there are new branches--perhaps a greater than expected number--could be useful information. For both the engineers and the operaters.

donkeyd 9y ago

How do you define a 'decision' in something like autopilot though? Also, if I remember correctly, only a couple of images per second are sent to Tesla. If there's a fraction of a second where it may decide to run into something, it may not actually appear in test results.

maxerickson 9y ago
One way is to extract the drive path for some given sensor input. Then you can classify the drive path as safe, marginal, collision, severe collision, etc. Changes from safe to safe aren't interesting. Changes in the wrong direction are obvious issues.

walrus01 9y ago

I really don't think they could do this effectively unless each car also uploads 20GB per day of 60fps high quality video, which would collapse the cellular networks they use.

makomk 9y ago

I don't think they have that much data that's useful for this purpose. They may have billions of miles of some kinds of data, but they only upload occasional samples of the camera video that would be required to test a feature of this sort. I can't find the exact link now, but someone reverse-engineered it and it was something on the order of a few seconds a few times an hour.

kalleboo 9y ago

The problem is you can't simulate the video that happens after your new decision. So this steering update would take you 0.3 degrees to the right. So what? Well those 0.3 degrees might change the angle of the line which influences the car to steer another 0.3 degrees, etc. But without that followup video (which you can't simulate from a 2D recording) you don't know how it would react to the change in environment. The only way to regression test these things is in a simulated 3D environment (or miles and miles of real test track)

darkerside 9y ago
Following the branch down is obviously impossible, but simply detecting that there are new branches--perhaps a greater than expected number--could be useful information. For both the engineers and the operaters.