3 ms·

I think you could argue that it isn't life-critical software because it requires active driver monitoring. What makes it different from lane-keeping assist in o

• jprete 3y ago
  The fact that it attempts to do more things can, in fact, make it more dangerous in the specific situations where a human interrupt is needed. Human attention works better the more it is needed, so if you need a human in the loop, you're best off with a system where their input is needed fairly often compared to a system where their input is needed once every long while. The other factor is that human attention may be needed in a situation where it is very hard to actively fix the situation if you're not already in the OODA loop. If you suddenly realize you need to override the car, and you're in the middle of a complex merge and the reason you need to override the car is because someone else screwed up, it may not be possible to solve the problem with the extra cost of your attention spinning up.
  • wilg 3y ago
    I understand what you're saying, but if you're not already in the OODA loop then you aren't actively monitoring the car. You could say, well, maybe the system lulls you into not being in that loop by taking most of the driving tasks away. I would say that having driven thousands of miles with it, it certainly doesn't feel like that to me. I'm still very, instinctively aware of how the car should be acting and even minor deviations from what I would do myself are very conspicuous and cause me to actively double-check its behavior. Also, in its current state, input is needed quite frequently. And the empirical data available (which could definitely be higher fidelity, granted) appears to suggest that the actual safety record is quite good. So we're left with a trade-off: there are some scenarios where a more featureful system could elevate risk, but there are other scenarios where it lowers them. It's not obvious on paper how to balance them, and the empirical (and, for me, experiential) data indicates the trade-off is a reasonable one. Perhaps a large amount of the concern around this comes from people not being aware of what it's like to operate a car in this mode. Maybe people think you really don't have to pay attention? It knows when you've got hands on the wheel, it knows when you're looking at your watch or your phone or the car's display, and it will pretty aggressively prevent you from getting complacent. Not only will it beep, it might lock you out from using it entirely. (Lockouts never happens to me, but I have seen tweets from it happening to others.) There was an old pre-Waymo Google talk about how partial driving assistance was dangerous for this reason, and I bought that argument but I no longer find it convincing having used such a system a lot.
• KaiserPro 3y ago
  > because it requires active driver monitoring. Yet it doesn't disengage when you take your hands off the wheel > What makes it different from lane-keeping assist in other vehicles? lane assist, assists you in keeping in the lane. It doesn't pretend to be able to drive for you. It also, as far as I can see, doesn't attempt to fling you into cars you're overtaking. (unless you don't use your indicators....) > bog-standard cruise control? That again isn't pretending to drive for you. The instructions are generally very clear, like on my car it says in huge text "this aid will not break for you, you must maintain distance as you normally would". Also, it doesn't attempt to steer you into a central reservation. There are shit cruise controls out there, like on the VW golf. That stops you if you attempt to drive past much slower traffic. > we accept that it is the driver's responsibility to not let that happen. Yes, because they have been told in no uncertain terms that if you obliterate someone in your car, its your fault and you're going to jail. More over, you were the one driving, so its down to you to not do it. Tesla are shipping untested systems and telling you, as an untrained roboticist supervisor: "don't worry, its safe, it drives it's self!" give you the impression that you don't really need to pay that much attention. > What is the difference here other than the system is _less likely_ to kill you because it is smarter? Its not that system. I will fully endorse a safe, tested, verifiable level 4 autonomous driving system. I will not endorse a barely credible level 2 system who's QA system involves YOLO deploying shit to unqualified customers.
  • wilg 3y ago
    > Yet it doesn't disengage when you take your hands off the wheel Well you don't want it to immediately disengage, which would be insanely dangerous, but it knows if your hands are on the wheel. It will progressively alert you more prominently if it hasn't felt wheel torque from your hands. It will then begin beeping loudly and stop and lock you out from using Autopilot. It also knows whether you are looking at the road from an internal camera and will beep if you are looking at your phone, watch, or the car's screen. This system works quite well. > lane assist, assists you in keeping in the lane. It doesn't pretend to be able to drive for you. It also, as far as I can see, doesn't attempt to fling you into cars you're overtaking. (unless you don't use your indicators....) "Pretending to drive for you" is meaningless. It's a feature that takes over driving functions that requires active supervision and that if you do not pay attention can cause a crash. The conditions of how it might cause that crash are not particularly important conceptually. What matters is that there is an expectation you pay attention or you might crash. The primary difference is cruise control is worse at driving. > Yes, because they have been told in no uncertain terms that if you obliterate someone in your car, its your fault and you're going to jail. More over, you were the one driving, so its down to you to not do it. Tesla are shipping untested systems and telling you, as an untrained roboticist supervisor: "don't worry, its safe, it drives it's self!" give you the impression that you don't really need to pay that much attention. This is also the case with FSD. You're told its your fault. You're told it's your responsibility. They do not say "don't worry it drives itself" or give you the impression that you don't really need to pay that much attention. In fact it says the following, among other things you must explicitly accept, enable, and agree to: > Full Self-Driving is in early limited access Beta and must be used with additional caution. It may do the wrong thing at the worst time, so you must always keep your hands on the wheel and pay extra attention to the road. Do not become complacent. When FSD is enabled your Model 3 will make lane changes off highway, select forks to follow your navigation route, navigate around other vehicles and objects, and make left and right turns. Use FSD in limited Beta only if you will pay constant attention to the road, and be prepared to act immediately, especially around blind corners, crossing intersections, and in narrow driving situations. Cabin Camera is used to detect driver attentiveness while FSD Beta is in-use to provide you with audible alerts, reminding you to keep your eyes on the road. > I will not endorse a barely credible level 2 system who's QA system involves YOLO deploying shit to unqualified customers. I'm not looking for your endorsement, I was trying to explain why the system is legal and probably less dangerous than you think. Also obviously they QA it before it goes out to "unqualified" customers.