3 ms·

This seems testable. IANAAE (I Am Not An Automotive Engineer), but why can't you run both the new and old code side by side and if the actions they take are mat

This seems testable. IANAAE (I Am Not An Automotive Engineer), but why can't you run both the new and old code side by side and if the actions they take are materially different investigate further? Like, if in one case the new code would want you to move left, and the old code goes straight, one of those behaviors is probably wrong. If the driver corrects, then the new code is probably correct, but if the driver does not, then the new code is doing something probably incorrect. At the very least, you should be able to get some sort of magnitude/fuzzy understanding of how frequently the new code is disagreeing, and you can figure out where and go check out those conditions.

jakecopp 8y ago
A/B testing self-driving car software to find bugs? Would probably work great, but that is also terrifying! But you're right, if you're really going to do a full roll out, may as well test it on a subsegment first - I'd hate for it to be used as a debugging tool though.
akira2501 8y ago
> but why can't you run both the new and old code side by side and if the actions they take are materially different investigate further? Because there are no closed facilities that you can use to actually perform any meaningful test. You could test "in-situ", but you would need an absolutely _huge_ testing area in order to accurately test and check all the different roadway configurations the vehicle is likely to encounter. You'll probably want more than one pass, some with pedestrians, some without, some in high light and some in low, etc.. It's worth noting that American's drive more than 260 billion miles each _month_. It's just an enormous problem.
- dchichkov 8y ago
  This particular case might have been testable "in-situ": [ABS News Reporter] Dan Noyes also spoke and texted with Walter Huang's brother, Will, today. He confirmed Walter was on the way to work at Apple when he died. He also makes a startling claim — that before the crash, Walter complained "seven-to-10 times the car would swivel toward that same exact barrier during auto-pilot. Walter took it into dealership addressing the issue, but they couldn't duplicate it there." It is very believable that the car would swivel toward the same exact barrier on auto-pilot. BTW - I'm running a nonprofit/public dataset project aimed at increasing safety of autonomous vehicles. If anyone here wants to contribute (with suggestions / pull requests / following it on twitter / etc) - you'd be most welcome. Its: https://www.safe-av.org/ https://www.safe-av.org/
- yogrish 8y ago
  This is where simulators play an important role. Many AD( automated driving) solution suppliers are investing into simulators to create different scenarios and test performance of their sensors and also SW. Else, as you said it’s impossible to drive billions of miles to cover all usecases.
salawat 8y ago
It has already been touched on by another commenter, but testability of Machine Learned systems outside of a training dataset is pretty much a crapshoot. An ML solution stops "learning" after training and only reacts. To illustrate the difference, have you driven on roads under construction lately? As humans, when you've driven the same road hundreds of times, you start to do the same thing as a machine learned implementation. You drive by rote. When you get to that construction zone though, or the lines get messed up, your brain will generally realize something has changed, and you'll end up doing something "unpredictable", i.e. learning a new behavior. The Machine Learning Ali's output (a neural net) can't do that. It can generify a pattern to a point, but it's behavior in truly novel circumstances cannot be assured. Besides which, the problem still stands that the system is coded to ignore straight ahead stationary objects. Terrible implementation. It should look for overly fast and uniform increase in angular field coverage combined with being near stationary in terms of relative motion as a trigger to brake. I.e. If a recognized shape gets bigger at the same rate on all "sides" whilst maintaining a weighted center at the same coordinate. It's one of the visual tricks pilots are taught to avoid mid-air collisions. Admittedly though, the human brain will likely remain WAY better at those types of tricks than a computer will be for a good long time.
- dominotw 8y ago
  Basically AI is a sophisticated ' hotdog not hotdog'.
- mortehu 8y ago
  I think the point is to run the two models on the same data, either simultaneously in everyone's cars, or using recorded sensor data from the past (maybe in the car while parked after a drive for privacy reasons). Initially, only the old version gets to take any action in the real world. Any difference in lane choice would then have to be justified before making the new version "live". You can do this sort of stuff when replacing a web service too, by the way. For example running two versions of Django and checking if the new version produces any difference for a week before making it the version the client actually sees.
  - 8y ago
taneq 8y ago
> why can't you run both the new and old code side by side and if the actions they take are materially different investigate further? As I understand it, this is essentially what they were doing with the autopilot 'shadow mode' stuff. Running the system in the background and comparing its outputs with the human driver's responses, and (presumably) logging an incident when the two diverged by any significant margin?

jakecopp 8y ago

A/B testing self-driving car software to find bugs? Would probably work great, but that is also terrifying! But you're right, if you're really going to do a full roll out, may as well test it on a subsegment first - I'd hate for it to be used as a debugging tool though.

akira2501 8y ago

> but why can't you run both the new and old code side by side and if the actions they take are materially different investigate further? Because there are no closed facilities that you can use to actually perform any meaningful test. You could test "in-situ", but you would need an absolutely _huge_ testing area in order to accurately test and check all the different roadway configurations the vehicle is likely to encounter. You'll probably want more than one pass, some with pedestrians, some without, some in high light and some in low, etc.. It's worth noting that American's drive more than 260 billion miles each _month_. It's just an enormous problem.

dchichkov 8y ago
This particular case might have been testable "in-situ": [ABS News Reporter] Dan Noyes also spoke and texted with Walter Huang's brother, Will, today. He confirmed Walter was on the way to work at Apple when he died. He also makes a startling claim — that before the crash, Walter complained "seven-to-10 times the car would swivel toward that same exact barrier during auto-pilot. Walter took it into dealership addressing the issue, but they couldn't duplicate it there." It is very believable that the car would swivel toward the same exact barrier on auto-pilot. BTW - I'm running a nonprofit/public dataset project aimed at increasing safety of autonomous vehicles. If anyone here wants to contribute (with suggestions / pull requests / following it on twitter / etc) - you'd be most welcome. Its: https://www.safe-av.org/ https://www.safe-av.org/
yogrish 8y ago
This is where simulators play an important role. Many AD( automated driving) solution suppliers are investing into simulators to create different scenarios and test performance of their sensors and also SW. Else, as you said it’s impossible to drive billions of miles to cover all usecases.

salawat 8y ago

It has already been touched on by another commenter, but testability of Machine Learned systems outside of a training dataset is pretty much a crapshoot. An ML solution stops "learning" after training and only reacts. To illustrate the difference, have you driven on roads under construction lately? As humans, when you've driven the same road hundreds of times, you start to do the same thing as a machine learned implementation. You drive by rote. When you get to that construction zone though, or the lines get messed up, your brain will generally realize something has changed, and you'll end up doing something "unpredictable", i.e. learning a new behavior. The Machine Learning Ali's output (a neural net) can't do that. It can generify a pattern to a point, but it's behavior in truly novel circumstances cannot be assured. Besides which, the problem still stands that the system is coded to ignore straight ahead stationary objects. Terrible implementation. It should look for overly fast and uniform increase in angular field coverage combined with being near stationary in terms of relative motion as a trigger to brake. I.e. If a recognized shape gets bigger at the same rate on all "sides" whilst maintaining a weighted center at the same coordinate. It's one of the visual tricks pilots are taught to avoid mid-air collisions. Admittedly though, the human brain will likely remain WAY better at those types of tricks than a computer will be for a good long time.

dominotw 8y ago
Basically AI is a sophisticated ' hotdog not hotdog'.
mortehu 8y ago
I think the point is to run the two models on the same data, either simultaneously in everyone's cars, or using recorded sensor data from the past (maybe in the car while parked after a drive for privacy reasons). Initially, only the old version gets to take any action in the real world. Any difference in lane choice would then have to be justified before making the new version "live". You can do this sort of stuff when replacing a web service too, by the way. For example running two versions of Django and checking if the new version produces any difference for a week before making it the version the client actually sees.
- 8y ago

taneq 8y ago

> why can't you run both the new and old code side by side and if the actions they take are materially different investigate further? As I understand it, this is essentially what they were doing with the autopilot 'shadow mode' stuff. Running the system in the background and comparing its outputs with the human driver's responses, and (presumably) logging an incident when the two diverged by any significant margin?