4 ms·

>This is actually the most serious aspect of all of these crashes: the system does not seem to be aware when it is getting things wrong. Although serious, this

>This is actually the most serious aspect of all of these crashes: the system does not seem to be aware when it is getting things wrong. Although serious, this is working as designed. Level II self-driving doesn't have automation that makes guarantees about recognizing scenarios it cannot handle. At level III, the driver can safely close their eyes until the car sounds the alarm that it needs emergency help. Audi plans to release a level III car next year, so we'll see how liability for accidents actually shakes out. Unfortunately level II is probably the most dangerous automation even with drivers that understand the limitations of the system. They still need constant vigilance to notice failures like this and react quickly enough to avoid collisions. Just imagine poorly marked or abrupt and ramps or intersections that drivers hardly have enough time to react when they're already driving. Add in the delay to notice the computer is steering you into a wall and yank the wheel can turn some of these accident prone areas into accident likely areas.

lisper 9y ago
> level II is probably the most dangerous automation I'll go further and say that level II is worse than useless. It's "autonomy theatre" to borrow a phrase from the security world. It gives the appearance that the car is safely driving itself when in fact it is not, but this isn't evident until you crash.
TaylorAlexander 9y ago
I disagree. It’s not as though level II is a hard and fast definition - Tesla wants level 4 and claims the cars possess the hardware for that already. So you’d think their level II would still be smart enough to detect these problems to some degree. It’s not a fixed system but one they keep upgrading. I would expect this nominally level II system to have more smarts than a system that is designed never to exceed level II.
- dragonwriter 9y ago
  > Tesla wants level 4 and claims the cars possess the hardware for that already. You can't possibly determine what hardware is required for Level 4 until you have proven a hardware/software combination, so that's just empty puffery, but even if it was true... > So you’d think their level II would still be smart enough to detect these problems to some degree. No, because the smartness of their system is about the software. They could have hardware sufficient to support Level 4 autonomy and better-than-human AI running software that only supports a less-thsn-adequate version of Level 2 autonomy. What their hardware could support (even if it was knowable) gives you no basis for belief about what their current software on it supports, except that it won't exceed the limits set by the hardware.

lisper 9y ago

> level II is probably the most dangerous automation I'll go further and say that level II is worse than useless. It's "autonomy theatre" to borrow a phrase from the security world. It gives the appearance that the car is safely driving itself when in fact it is not, but this isn't evident until you crash.

TaylorAlexander 9y ago

I disagree. It’s not as though level II is a hard and fast definition - Tesla wants level 4 and claims the cars possess the hardware for that already. So you’d think their level II would still be smart enough to detect these problems to some degree. It’s not a fixed system but one they keep upgrading. I would expect this nominally level II system to have more smarts than a system that is designed never to exceed level II.

dragonwriter 9y ago
> Tesla wants level 4 and claims the cars possess the hardware for that already. You can't possibly determine what hardware is required for Level 4 until you have proven a hardware/software combination, so that's just empty puffery, but even if it was true... > So you’d think their level II would still be smart enough to detect these problems to some degree. No, because the smartness of their system is about the software. They could have hardware sufficient to support Level 4 autonomy and better-than-human AI running software that only supports a less-thsn-adequate version of Level 2 autonomy. What their hardware could support (even if it was knowable) gives you no basis for belief about what their current software on it supports, except that it won't exceed the limits set by the hardware.