6 ms·
> And it's not much of a problem if every human in a semi-autonomous vehicle followed the automakers' explicit, insistent instructions to pay attention at all t
by tomxor 9y ago
> And it's not much of a problem if every human in a semi-autonomous vehicle followed the automakers' explicit, insistent instructions to pay attention at all times, and take back control if they see a stationary vehicle up ahead.
While their technical reasoning is fine... this statement shows a complete failure of reasoning about human beings and UX. In essence they are asking people to stay alert without being active, this is actually really fucking hard to do as a human being. Noticing an object that needs to be avoided while you are already actively engaged in driving is easy, you are naturally alert and there is no "intervention time" because there is nothing to intervene with. Trying to achieve the same level of response while letting a machine drive kind of defeats the purpose of letting the machine drive.
Yes I'm basically making the argument that semi-autonomous driving is not safe. Make it fully autonomous or not at all. Two modes, keep it simple, because human concentration is complex.
- jacquesm 9y agoThere are mountains of evidence to back this up, quite a bit of it stems from the problem of keeping airline pilots engaged and alert while the autopilot functions as it should.
- ghaff 9y agoAnd people deskill when they don't get enough practice as well. I know I got better over time as a driver and it wasn't just because I wasn't a teenager any longer. It's easy to imagine we'll end up with a lot of people on the roads with not-quite autonomous systems who actually haven't actively driven all that much.
- rsynnott 9y agoApparently some airlines now encourage pilots to do certain tasks that the autopilot can and normally would do from time to time, to avoid this.
- aaronmdjones 9y agoCorrect. The autopilot is fully capable of landing the plane entirely automatically (once you tell it to, and if you're approaching an ILS autoland runway), but often times that's reserved for extremely poor weather or such, and pilots will land manually. They'll still tune the ILS so they have their localiser and glideslope information on their flight display, but they'll follow that manually rather than telling the autopilot to. This varies by airline. On Boeing airliners there's also the (delivery) option for a pilot awareness monitor. If you don't make any inputs (change from weather to terrain radar or vice versa, change the range of information displayed on your navigation display, toggle whether airports or VORs are displayed, etc) for a short period of time (I believe it's 10 minutes), it will generate an EICAS warning. This is attention-grabbing yellow text on a screen that the pilots are supposed to be monitoring. If you don't action that (by hitting the warning reset switch or making any input) you get a very loud very red alarm.
- tialaramex 9y agoNote that alertness checks are (somewhat) effective in commercial aviation because the industry AND the pilots accept them as normal. A big ship will have BNWAS, which is supposed to alert everybody else in crew quarters (e.g. officers asleep) if the officer on the bridge (who should be actively controlling it) doesn't do anything for a prolonged period and doesn't respond to a local warning alarm. When there's an accident investigation with suspicious circumstances like a collision in an otherwise empty region in the middle of a calm dark night, almost invariably either the BNWAS was "mistakenly" switched off, or its mandatory logging was "mistakenly" erased or turns out never to have been enabled at all. Why? Because the alarm is annoying and there's no acceptance by owners, masters or crews that it's keeping them alive and so they must use it, so they switch it off, muffle the alarms or otherwise disable it and then, inevitably, someone falls asleep and an accident happens. Similar tech in private cars would, I think we have to assume, end up just like BNWAS on big freighters, disabled because it was annoying. Sometimes nobody would survive the accident to cover up the fact it was disabled, but that's cold comfort.
- ghaff 9y ago
- KallDrexx 9y agoI've noticed my driving has gotten a lot less smooth now that I moved near a train station and take the commuter train into work every day. I drive about once a week now and my acceleration and braking has definitely got a bit more jerky.
- patcheudor 9y agoIt's not even about deskilling. I raced road bikes for years and in large part drive like I rode: highly defensive (bordering on aggressive as I've been lectured by LEOs). I commuted for years in my roadster with a manual transmission & power steering set for the track where I'm highly engaged as I can feel every crack in the road and the car can accelerate to unholy speeds in seconds. When I drive our sedan, it's boring and I find my mind drifting. If I turn on assisted cruise control, it's mind numbingly boring and on any long drive I'm guaranteed to fall asleep so I don't use it anymore because it's too dangerous. Without being engaged in the act of driving, without the adrenaline, I switch off. This is the key problem I suspect everyone at some level suffers from wether they want to admit it or not. Even awake, on full autopilot I suspect most people are in full-on daydream mode.
- tomxor 9y agoI don't race but I have noticed this with my "normal" car, i've always driven a hatchback though which are quite common in this country. If I force myself to drive very steady I can almost feel the alertness drifting away, I become less aware of vehicles around me and their bearing. If I speed up to a threshold just between the bored comfortable zone and racing it's enough to give heightened alertness, I can easily preempt drivers making careless moves that would otherwise endanger me and I notice pedestrians and other things around the road more, it's a strange balance, but i've no doubt that sticking dead to the limit on a large road will decrease alertness in any study.
- FabHK 9y agoAlso called "Children of the Magenta" (the cues provided by the flight management computer on the primary flight display are typically displayed in magenta). Really a conundrum - automation forestalls many accidents, but pilots get less hands-on experience and are less prepared to cope when the automation fails. Excellent related talk by Capt. Warren VanderBurgh of American Airlines from 1997 (!): https://www.youtube.com/watch?v=pN41LvuSz10 https://www.youtube.com/watch?v=pN41LvuSz10 EDIT to add: some background: https://99percentinvisible.org/episode/children-of-the-magenta-automation-paradox-pt-1/ https://99percentinvisible.org/episode/children-of-the-magen...
- omarforgotpwd 9y agoAnd we all know how much damage that has caused the airline industry! Oh wait, there wasn't a single death on a US passenger airline last year, a record low in the history of aviation.
- jacquesm 9y agoI really don't think the discipline of the average driver comes even close to the discipline of the average airline pilot. That, and very very strict oversight of every aspect from the design of aircraft all the way to maintenance is what aviation safe.
- dominotw 9y ago> In essence they are asking people to stay alert without being active, this is actually really fucking hard to do as a human being. This problem has been studied to death in the airline industry. Surprising to see this repeated all over again.
- twoodfin 9y agoFWIW, I find "semi-autonomous" features like automatic cruise control & lane keeping to be a net benefit to my alertness on long trips as they reduce fatigue from constant minor adjustments in speed and direction. Clearly it's possible to use these features in a way that increases risky distraction, but I think it's a mistake to assume such a driving mode can't be safer on net when used properly.
- piva00 9y agoI have the opposite experience with lane keeping, it made me space out in a way that when lane keeping didn't work properly I was really scared on how long it took me to react. Definitely vouch for cruise control as having a cramping leg stuck on the accelerator for 5+ hours driving sucks but, personally, would say otherwise for lane keeping.
- CodeCube 9y agoThough I use it for the same reason you mention ... I always am paranoid when I have cruise control engaged for very long distances. Have no idea how likely this would be, but the fear that creeps and stays in the back of my head is that I'll get comfortable, reposition my feet as I adjust for comfort, and then miss the brakes in the event of an emergency
- andrewfong 9y agoMy favorite semi-autonomous feature is the one where your car detects a stopped vehicle in front of you and tells you to brake.
- theluketaylor 9y agoI rarely use even basic cruise control as I find it extremly hard to stay engaged with it enabled. Small throttle adjustments seem to be the trigger for my driving routines of checking mirrors, monitoring gaps and watching the horizon for curves. Just making steering adjustments isn't enough to keep my brain engaged; I zone out don't maintain enough situational awareness. I can easily imagine there are people who can use lane keeping and radar cruise control safely, but I also know I am not one of them. I'll need to stick with manual driving until the car can drive itself in all cases.
- dustingetz 9y agoThey are fully autonomous. It's just legalese to shift liability to the consumer
- EADGBE 9y agoThen why have a steering wheel at all?
- qbrass 9y agoYou can't blame the human behind the wheel if there's no steering wheel.
- EpicEng 9y ago>but its manual does warn that the system is ill-equipped to handle this exact sort of situation: “Traffic-Aware Cruise Control cannot detect all objects and may not brake/decelerate for stationary vehicles, especially in situations when you are driving over 50 mph (80 km/h) and a vehicle you are following moves out of your driving path and a stationary vehicle or object is in front of you instead. Yeah that definitely sounds 100% autonomous... ship it!
- ghaff 9y agoYes. There's decades of research on this. Some level of automation probably does help. It's not like people don't zone out, especially for freeway driving, even when they're driving with totally manual systems. And, of course, people text, fiddle with radios, etc. But too much automation that's not totally for backup may well be harmful.
- mnw21cam 9y agoIt's still illegal to text or otherwise use a handheld phone while driving, even with the automation that we have on some cars now. I imagine this will be the case until the automation is so complete that there is no longer a steering wheel. I still see people driving around while holding phones to their ears. There needs to be a change to make it socially unacceptable to do this. You only need to be caught twice to lose your license. It's also illegal to drive distracted. While it's technically legal to use a hands-free telephone, if you could be described as "distracted" because of it, it's illegal.
- 51Cards 9y agoI don't think the statement fails in that it's accurate of the expectations. What has failed is the understanding of the manufacturers of, as you said, human concentration. They advertise semi-autonomous driving as making driving easier, but you still demand the customer invest 90% of the same effort in staying alert. I still have to concentrate, be alert, and keep a hand on the wheel. Literally all I'm "saving" is several hundred minute muscle movements to keep the car aligned. The "semi" part so far out-weighs the "autonomous" part the term (to me) really doesn't fit.
- ksk 9y ago>While their technical reasoning is fine... this statement shows a complete failure of reasoning about human beings and UX. In essence they are asking people to stay alert without being active, this is actually really fucking hard to do as a human being. This is the common case in pretty much all automated manufacturing plants where you have to hit the big red STOP button when stuff goes wrong. I don't quite see how its a failure when it has been working for them. In any case, do you have an alternative proposal that works?
- EpicEng 9y agoI think the alternate proposal is that semi-automated driving is a bad idea and shouldn't be a thing. It's not on an HN poster to solve the problem for Tesla.
- ksk 9y agoIts also bizarre to identify problems, and assume the mantle of a teacher and tell someone they're doing it wrong without having any first hand expertise.
- EpicEng 9y agoIt's not bizarre at all when you have decades of research on your side. We know that taking the human out of the equation reduces their response times and awareness dramatically. This isn't a new area of research.
- loceng 9y agoIf people can't focus or concentrate enough to pay attention to what's important, shouldn't they have the responsibility to pull over until they are able to do what's insisted? The same would go for if they're in full control of a vehicle however they're falling asleep, we'd expect them to pull over? Perhaps Tesla, if the model has a camera, can detect where the drivers are - and if not consistently looking forward then alert them?
- tzakrajs 9y agoThe benefit of autonomous driving is not having to pay attention. If I have to pay attention to the highway I may as well be in my Ford Taurus with my steering wheel on my leg and my hands free. How about I save $$$$$ on huge rechargable batteries and I get laser sensors in exchange. EV and self driving should be decoupled.
- loceng 9y agoObviously Tesla's AutoPilot doesn't satisfy what you're wanting - but you're still capable of understanding if you're instructed that AutoPilot still requires your full attention then that means you need to always pay attention, right?
- omarforgotpwd 9y agoYeah, I have a Tesla with AutoPilot. Staying alert enough to make sure it doesn't crash into anything really isn't that hard. It's still really nice to not have to worry about keeping in the lane and maintaining distance yourself, but you can easily get a sense for when the car is doing something you wouldn't do. There have been many times when I'm pretty sure AutoPilot was about to kill me so I just... hit the breaks. Trust me, it's still a lot safer than if I was driving the car. Naturally there is always going to be human error, but it doesn't make sense to say "semi-autonomous" is bad while "full autonomy" is good. Guess what, we're going to have to deal with less advanced versions of the software before we can arrive at the real deal. The trick is to know and communicate the limits of the system. For example Tesla will alert you visually and through audio if you don't have your hands on the wheel. If it has to ask you three times, it'll just disable AutoPilot for the rest of the drive. It's just like a more advanced cruise control. If I hit a fire truck because I was on cruise control would anyone write an article blaming cruise control? No, because everyone knows the limits of that system. Crashes like this will force people to learn the limits of current autonomous software as well.
- panarky 9y agoThat's why Google insists that full autonomy is required. They don't think semi-autonomy is safe enough. Better to completely remove the steering wheel, brakes and gas pedal than try for some half-way hybrid. No human can stay alert enough to take split-second evasive maneuvers while doing nothing 3599 seconds out of 3600.
- dwild 9y agoThis is absurd, it's exactly the same argument as when cars were invented, people can't drive thus it will kill more people!!! You know what's the solution? Don't use the feature if you can't use it. There are people that drink and drive, we didn't ban cars because of that... If you can't achieve the same level of awareness or don't want to, well, too bad, don't use it. My SO stress when she drives, she is a good driver but she just doesn't like it. I'm pretty sure she will have enough awareness, but that thing will lower her stress considerably and will make driving much more pleasant for her.