3 ms·

How can you measure the performance of an ML engine in untested conditions? Tesla's whole goal is to achieve L5 capabilities. That means anywhere in any conditi

How can you measure the performance of an ML engine in untested conditions? Tesla's whole goal is to achieve L5 capabilities. That means anywhere in any conditions, which is clearly impossible to comprehensively validate with a finite test set. It's plausible that we could get a long period of acceptable safety before a black swan event suddenly causes deaths because it can't handle smoke, heavy rain/fog, or blizzards.

ajross 5y ago
That's just ludditism. That logic works for any new technology. How do you know vaccines are safe under all conditions? How do you know planes won't crash under some "black swan" event? How do you know GM food won't poison you? How do you know your fuel tank won't explode? How do you know your ocean liner won't hit an iceberg and sink? There's literally no technology in the world that meets your standards. You don't know. You measure and you decide, based on numbers and science and moral reasoning about risk. You don't get absolutes from anything else, why are you demanding it from cars?
- AlotOfReading 5y ago
  I'm not sure what in my post gave you the impression that I'm demanding perfection, but it's incorrect. My interpretation of the GP's point is that we know these kinds of perception models have very real failure cases in the presence of noise and adversarial images in the environment, among other possible operating conditions. They're asking what safeguards exist to mitigate the risk of a system failure, which is something other SDC companies have put enormous amounts of work into. It's a perfectly fair question and the general expectation of things we have as a society is that things generally shouldn't fail catastrophically at random. In a plane, that's achieved to socially acceptable levels by an incredible amount of systems engineering, redundancy, aerodynamics, and ultimately well trained pilots & ATC. With food it's achieved by government oversight, standards, and the general principle that most things won't harm you. With ships, we simply have them avoid icebergs. What isn't a good answer is just handwaving about measuring model performance. That's the smallest of the many prerequisites to safety here.
jungturk 5y ago
From the video, it looks like the team is trying to minimize the set of untested conditions through simulation and clips from the fleet. I agree with you that those techniques won't eliminate errors, but I don't think that's the requirement for L5 - human drivers' errors cause lots of deaths today.

ajross 5y ago

That's just ludditism. That logic works for any new technology. How do you know vaccines are safe under all conditions? How do you know planes won't crash under some "black swan" event? How do you know GM food won't poison you? How do you know your fuel tank won't explode? How do you know your ocean liner won't hit an iceberg and sink? There's literally no technology in the world that meets your standards. You don't know. You measure and you decide, based on numbers and science and moral reasoning about risk. You don't get absolutes from anything else, why are you demanding it from cars?

AlotOfReading 5y ago
I'm not sure what in my post gave you the impression that I'm demanding perfection, but it's incorrect. My interpretation of the GP's point is that we know these kinds of perception models have very real failure cases in the presence of noise and adversarial images in the environment, among other possible operating conditions. They're asking what safeguards exist to mitigate the risk of a system failure, which is something other SDC companies have put enormous amounts of work into. It's a perfectly fair question and the general expectation of things we have as a society is that things generally shouldn't fail catastrophically at random. In a plane, that's achieved to socially acceptable levels by an incredible amount of systems engineering, redundancy, aerodynamics, and ultimately well trained pilots & ATC. With food it's achieved by government oversight, standards, and the general principle that most things won't harm you. With ships, we simply have them avoid icebergs. What isn't a good answer is just handwaving about measuring model performance. That's the smallest of the many prerequisites to safety here.

jungturk 5y ago

From the video, it looks like the team is trying to minimize the set of untested conditions through simulation and clips from the fleet. I agree with you that those techniques won't eliminate errors, but I don't think that's the requirement for L5 - human drivers' errors cause lots of deaths today.