17 ms·

>t's a tool to help attentive drivers avoid accidents that might have otherwise occurred. This needs far more discussion. I just don't buy it. I don't believ

>t's a tool to help attentive drivers avoid accidents that might have otherwise occurred. This needs far more discussion. I just don't buy it. I don't believe that you can have a car engaged in auto-drive mode and remain attentive. I think our psychology won't allow it. When driving, I find that I must be engaged and on long trips I don't even enable cruise control because taking the accelerator input away from me is enough to cause my mind to wander. If I'm not in control of the accelerator and steering while simultaneously focused on threats including friendly officers attempting to remind me of the speed limit I space out fairly quickly. In observing how others drive, I don't think I'm alone. It's part of our nature. So then, how is it that you can have a car driving for you while simultaneously being attentive? I believe they are so mutually exclusive as to make it ridiculous to claim that such a thing is possible.

cdblades 9y ago
This same thing can be said of auto-pilot in planes. Not saying it invalidates your argument, just that there are pretty wide-reaching consequences to that idea.
- mrguyorama 9y ago
  Which is why pilots are still in control of the plane while autopilot is active. Even while active, there is still a "pilot flying" and pilots are still responsible for scanning gauges and readouts and verifying the autopilot is doing what they expect. They do not just turn on autopilot and goof off
  - netc 9y ago
    they are expected to remain in control of the plane, not necessarily they do: https://timesofindia.indiatimes.com/india/Pilots-sleep-as-flight-attendant-turns-off-autopilot-on-Bangkok-Delhi-flight/articleshow/19860300.cms https://timesofindia.indiatimes.com/india/Pilots-sleep-as-fl...
  - Johnny555 9y ago
    Just like Tesla's auto pilot? Drivers are supposed to be "flying", scanning gauges and the road ahead to ensure the autopilot is doing what they expect... I think people that say that "autopilot" is a bad name for this feature don't really understand what an "autopilot" does.
    - mannykannot 9y ago
      An airplane under autopilot is not a second or two away from a fatal crash.
      - Johnny555 9y ago
        If the pilots are not flying, then it can be just a short time away from a crash. Like when the pilot is not paying attention and by the time the auto pilot can no longer fly, the pilot doesn't have enough situational awareness to take over. http://www.slate.com/blogs/the_eye/2015/06/25/air_france_flight_447_and_the_safety_paradox_of_airline_automation_on_99.html http://www.slate.com/blogs/the_eye/2015/06/25/air_france_fli...
        mannykannot 9y ago
        That is very much an outlier, and if it were at all relevant to the issue it would further weaken your case, as these three pilots had several minutes to sort things out. Questioning the assumptions underlying the assumed safety of airplane autopilot use can only weaken the claim that Tesla's 'autopilot' is safe.
      - mulmen 9y ago
        The fatal impact may not be seconds away but the event that sets in motion the series of actions that results in that fatal impact may take only seconds.
        mannykannot 9y ago
        The issue is how long between the problem first manifesting itself and a crash becoming inevitable. Note that even as short a time as ten seconds is an order of magnitude longer than one second. There are at most only a few rare corner cases in aviation where proper use of the autopilot could take the airplane within a minute of an irretrievable situation that would not have occurred under manual control.
  - stale2002 9y ago
    Uhhhhh, I think you are misinformed. They aren't going to literally fall asleep, but much of the time pilots are reading a book and not directly paying attention in the same way that a driver is.
- krrrh 9y ago
  This comparison really only works if you mean engaging autopilot with a bunch of other planes flying in close formation and if clouds were made of steel and concrete. Most of the time that autopilot is engaged on a flight there is next to zero risk of a collision. Commercial pilots also get a lot of training specifically related to what autopilot is and isn’t appropriate for.
  - laythea 9y ago
    That is a valid counterpoint in my opinion.
  - nightski 9y ago
    Even as a private pilot you are taught to continuously scan your instruments and surroundings as you fly through the air with autopilot enabled. Flight training is much more extensive than driving tests (although still not too bad) and they really drive procedures into you.
- skrause 9y ago
  Planes don't suddenly crash into a mountain because you have looked away for two seconds, there is much more time to react in case the autopilot doesn't behave correctly.
  - z3t4 9y ago
    once you see the mountain it will be too late. think of big airplannes as trains.
  - ptaipale 9y ago
    They can very suddenly crash into another plane because you have looked away for two seconds. It has actually happened (though nowadays, technology has made it easier to avoid these accidents).
    - AnimalMuppet 9y ago
      Planes are supposed to maintain a 5 mile separation distance. You aren't going to break that down in two seconds. (But head on, with both planes traveling 600 MPH, you can do it in 15 seconds. But both pilots would have to be inattentive for that time.)
      - ptaipale 9y ago
        They are supposed to, but if flight control doesn't help them do it, pilots have only very little time to react to what they meet. This was demonstrated in the Hughes Airwest collision with an F-4 in 1971. Since then, air traffic control procedures have been improved to avoid these situations, but nowadays e.g. over the Baltic Sea, Russian military planes are routinely flying with their transponder turned off so that flight control does not know where they are. So, this risk is still there. https://en.wikipedia.org/wiki/Hughes_Airwest_Flight_706#Probable_cause_and_recommendations https://en.wikipedia.org/wiki/Hughes_Airwest_Flight_706#Prob...
fapjacks 9y ago
I have to preface my post to say that I think developing self-driving automobiles is so important that it's worth the implied cost of potentially tens of thousands of lives in order to perfect the technology, because that's what people do; make sacrifices to improve the world we live in so that future generations don't have to know the same problem. But I think you're right. I think the "best" way to move forward until we have perfected the technology is not something that drives for you, but something that will completely take over the millisecond the car detects that something terrible is about to happen. People will be engaged because they have to be engaged, to drive the car. The machine can still gather all the data and ship it off to HQ to improve itself (and compare its own decisions to those of the human driver, which IMO is infinitely more valuable). But if there's one thing the average person is terrible at, it's reacting quickly to terrible situations. You're absolutely right that people can't be trusted to remain actively engaged when something else is doing the driving. Great example with the cruise control, too.
- rightbyte 9y ago
  Thats kinda Volvo Cars approuch, word by word, as one of their RnD engineers explained it to me.
  - southern_cross 9y ago
    IIRC, it was also Volvo who a few years back said that they would gladly take on any liability issues for their self-driving cars. Only to backtrack on that a short while later after having learned what liability laws in the U.S. actually look like, saying that they wouldn't take on such liability until the laws are changed to be more in their favor. So there's that ...
- almost_usual 9y ago
  > I have to preface my post to say that I think developing self-driving automobiles is so important that it's worth the implied cost of potentially tens of thousands of lives in order to perfect the technology, because that's what people do; make sacrifices to improve the world we live in so that future generations don't have to know the same problem. I generally agree with this philosophy but this is very optimistic, at least in the United States. This is a country where we can't even ban assault rifles let alone people from driving their own vehicles. You're going to see people drive their own vehicles for a very long time even if self driving technology is perfected.
iro3939 9y ago
I think our psychology has been manipulated such that we ignore 1 million deaths a year by humans and don’t at all take seriously halting car sales But it hasn’t happened to me! says a rando not dealing with the death of a loved on or financial difficulty Gotta ship!
hippich 9y ago
Quite a lot years ago I had to drive for 6 hours straight at night to get on time to place where I needed to be (air flight was not available back then for me) What I noticed that when I was following posted/safe speed limit, I was quickly losing focus, mind started wandering and eventually I felt I was falling asleep. I do not remember what made me to speed up, but once I was about 30% faster than posted speed limits, and once I reached part of the way where road was quite bad + a lot of road work was happening, I realized that I much more alert. As soon as I slowed down to posted speed limit speed I began drifting away again.. If anything, my anecdote confirms your theory - as soon as we perceive something safer, we pay way less attention. And Autopilot sounds like one of these safety things, which makes drivers less attentive and potentially missing dangerous situation, which otherwise would be caught by driver's mind. I wonder if there is a way to introduce autopilot help without actually giving sense of security to the driver. Granted Tesla would lose so precious marketing angle, but if their autopilot would work somewhat like variable power steering system on background without obvious taking over control of the car, in the long haul that would be more beneficial?
- pbhjpbhj 9y ago
  Assuming you were in a vehicle with ICE speed relates to vibration/noise which can, at the wrong pitch, cause drowsiness easily. This is why parents will take their children out in the car if a child is not sleeping well. I find rough motorway surfaces in my current vehicle induce heavy drowsiness at motorway speed limits (slight reduced at marginally higher speeds when the pitch is higher).
  - hippich 9y ago
    This is interesting idea, never thought about it! thanks!
chaostheory 9y ago
> This needs far more discussion. I just don't buy it. I don't believe that you can have a car engaged in auto-drive mode and remain attentive. I think our psychology won't allow it. Does anyone know of psychology studies that measure human reaction time and skill when sometime like autopilot is engaged most of the time? I remember taking part in a similar study at Georgia Tech that involved firing at a target using a joystick. It was also simultaneously a contest because only the top scorer would get prize money. The study was conducted in two parts. In the 1st phase, the system had autotargeting engaged. All subjects had to do was press a button when the reticle was on the target in order to score. In the 2nd phase, which was a surprise, autotargetting was turned off. I won the contest and my score was miles ahead of anyone. I can't fully confirm it but I feel this happened because I was still actively aiming for the target even when autotargetting was active.
- Animats 9y ago
  Does anyone know of psychology studies that measure human reaction time and skill when sometime like autopilot is engaged most of the time? Yes. That's been much studied in the aviation community.[1] NASA has the Multi-Attribute Test Battery to explicitly study this.[2] It runs on Windows with a joystick, and is available from NASA. The person being tested has several tasks, one of which is simply to keep a marker on target with the joystick as the marker drifts. This simulates the most basic flying task - flying straight and level. This task can be put on "autopilot", and when the marker drifts, the "autopilot" will simulate moving the joystick to correct the position. But sometimes the "autopilot" fails, and the marker starts drifting. The person being tested is supposed to notice this and take over. How long that takes is measured. That's exactly the situation which applies with Tesla's "autopilot". There are many studies using MATB. See the references. This is well explored territory in aviation. [1] http://web.ics.purdue.edu/~archerj/Course%20Deliverables/IE%20590%20(AAE%20590)%20-%20Effects%20of%20Automation%20in%20the%20Aircraft%20Cockpit%20Environment.pdf http://web.ics.purdue.edu/~archerj/Course%20Deliverables/IE%... [2] https://matb.larc.nasa.gov/software/ https://matb.larc.nasa.gov/software/
  - chaostheory 9y ago
    This sounds identical to the study I participated in except it was conducted by Georgia Tech and not NASA.
cdancette 9y ago
For me it's the opposite. I feel like with cruise control on, I'm much more attentive to the road and the surroundings and feel much safer. It really annoys me when I have to constantly look at the speed to avoid getting above the limit and it is very tiring for long drives.
- greglindahl 9y ago
  I totally agree with this sentiment -- the only reason I drive a safe speed is that I use cruise control constantly. Then I don't have to think about speed, and I can focus on everything else.
- kolpa 9y ago
  You need a speed limiter, not necessarily cruise control.
  - dhimes 9y ago
    Cruise control works fine for this.
  - greglindahl 9y ago
    A speed limiter involves me having to continue to press on a pedal. That's quite different from cruise control.
- flashmob 9y ago
  Noticed this also, no need to monitor and adjust the speed which is a mundane task (in cruise control traffic conditions). Eyes can be on the road instead. This is similar to the problem for pilots, who can be distracted by mundane tasks due the complexity of controls in modern aircraft. If these tasks are removed, the pilot can focus on what's more important. According to NASA " For the most part, crews handle concurrent task demands efficiently, yet crew preoccupation with one task to the detriment of other tasks is one of the more common forms of error in the cockpit." https://asrs.arc.nasa.gov/publications/directline/dl10_distract.htm https://asrs.arc.nasa.gov/publications/directline/dl10_distr...
- 9y ago
cyberferret 9y ago
I agree with the psychology aspect of driving. I've seen it mentioned many times that a large majority of auto accidents occur a few minutes from the driver's home, and usually on their way home. Apparently, being close to their neighbourhood and in familiar surrounds, the driver's attention tends to wane as they get distracted with other things that they have to do when they get to their house. Racing drivers have also reported that when they are not driving at 100%, they are more prone to make mistakes or crash. Most famously, Ayrton Senna's infamous crash at Monaco when he was leading the field by a LONG way. When he was asked why he crashed at a fairly innocuous slow corner, he said that his engineer had asked him over the radio to 'take it easy' as there was no chance he would be challenged for 1st place before the finish line, so he relaxed a fraction and started thinking about the victory celebrations. And crashed.
- intended 9y ago
  I thought the point was that most car related jaunts are short ones.
- kazinator 9y ago
  Accidents take place close to home because driving takes place close to home and accidents take place where driving takes place. Basically this: https://xkcd.com/1138/ https://xkcd.com/1138/
kafquaesque 9y ago
If a person uses a device while using autopilot, (which seems highly likely—not sure in thi instance) wouldn’t it be advisable to have the alerts come to them directly on whichever device they are using? The alert breaks them out of whatever task they are focusing on. If the alert is coming from the car I can see how a lot of us could ignore it.
- ehnto 9y ago
  What a world. Where we can't even take enough responsibility to be present enough to hear the "You are about to die" bell chime. Imagine the possible breaking components in that chain too. Bluetooth can fail, satellite can fail, cell can fail, WiFi can fail, a USB cable can fail, there isn't a single piece of connectivity technology that would make me confident enough to delegate alerts to another device. There is also an inherent failure of alarms in general in that even very loud ones can be ignored if they give false positives even once or twice. There is a body of study trying to address it. Some of the most fatal industrial accidents occured because alarms were either ignored or even fully switched off. We aren't good with alarms. I think the meat of it though is that unless Auto-pilot works perfectly then you can't leave it alone. And if you can't leave it alone then what's the point? The sell for autonomous cars isn't that people are just so darn tired of turning the steering wheel that they would really rather not. It's that we could potentially be more productive if we could shift our full attention to work/study/relaxation while commuting.
  - kafquaesque 9y ago
    It seems like we are in an “in-between” state where we are using humans to assuage the fears of people that aren’t sure if neural networks can drive better than humans. The goal is to eventually focus on something else. If it’s just about making driving safer I would think it’s more of an incremental innovation step versus the breakthrough concept of being able to do something else while being commuted completely by a neural-network driven vehicle. The bridge to get to this breakthrough hopefully isn’t hacked apart by naysayers. Every death should be met with empathy and a desire to strengthen this bridge and quicken the speed of crossing it.
hyperbole 9y ago
I don't buy this either nor should we it's not how the feature is marketed. "The system is designed to be able to conduct short and long distance trips with no action required by the person in the driver’s seat." The result of this statement and the functionality that matches it is it creates a re-enforced false sense of security. Does it matter whether the driver of the model X whose auto pilot drove straight into a center divider had his hands on the wheel if the outcome of applying autopilot is drivers focus less on the road? What is the point of two drivers one machine one human? You cannot compare car auto pilot to airplane they're not even in the same league. How often does a center divider just pop up at 20k ft? Usually machinery either augments human capabilities by enhancing them, or entirely replaces them. This union caused by both driver and car piloting the vehicle has no point especially when it's imperfect. I'm not opposed to Tesla's sale of such functionality, sell whatever you want, but I am opposed to the marketing material selling this in a way that contradicts the legal language required to protect Tesla... There's risks in everything you do, but don't market a car as having the hardware to do 2x your customers driving capability and then have your legal material say: * btw don't take your hands off the steering wheel... especially when there's a several minute video showing exactly that. Tesla customers must have the ability to make informed choices in the risks they take.
- jameslevy 9y ago
  There is a solution for this - if the driver shows any type of pattern of not using the feature safely, disable the feature. Autopilot and comparable functionality from other vehicles should be considered privileges that can be revoked.
  - hyperbole 9y ago
    Systems that are semi autonomous where there's some expectation of intervention work well in those scenarios, keep the car in lane markers on the highway, etc. Make sure the users hands are on the wheel but for fully autonomous even if your hands are on the wheel, how does it know your paying attention?
    - 9y ago
paganel 9y ago
> When driving, I find that I must be engaged and on long trips I don't even enable cruise control because taking the accelerator input away from me is enough to cause my mind to wander. Glad I'm not the only doing it. When driving on a highway I increase or decrease my car's speed by 10-15 km every 10 minutes or so, so that this variation can help me keep attentive about my surroundings.
Retric 9y ago
Your belief is meaningless, we have hard data that shows a net benefit to these systems. It's not a question of zero deaths, it's a question of reducing the number which means you need to look beyond individual events. Remember the ~90 people who died yesterday from a US car accident without making the news are far more important than a few individuals.
- phkahler 9y ago
  >> our belief is meaningless, we have hard data that shows a net benefit to these systems. No we don't. Tesla likes to compare their deaths per mile to the national average. The problem is that their autopilot is not fit to drive everywhere or in all conditions that go into that average. There is no data to support that autopilot is safer overall. It may not even be safer in highway conditions given that we've seen it broadside a semi and now deviate from the lane into a barrier - both in normal to good conditions.
  - Retric 9y ago
    Specific failures are again meaningless. Computers don't fail the way people do, on the other hand people also regularly fall asleep at the wheel and do similar dumb things. And really, driving conditions are responsible for a relatively small percentage of vehicle fatalities. Most often it's people doing really dumb things like driving 100+ MPH. The only thing we actually know is these cars are safer on average than similar cars without these systems. That's not looking at how much they are used, just the existence of said safety system and likely relates to them being used when the drivers are extremely drunk or tired which are both extremely dangerous independent of weather conditions.
    - kalleboo 9y ago
      So how about we adopt much cheaper and simpler solutions like drowsiness detection (Volvos have these), automatic emergency braking (I think every brand has this as an option now), breathalizer locks, speed limiters etc? The US just mandated all new cars have backup cameras, but it seems like mandating AEB would make a bigger difference.
pavanred 9y ago
I suppose its perhaps how they marketed the feature. We have parking assist feature in many cars, there's a reason its not auto-park instead. If it really was a feature to help attentive drivers avoid accidents, it probably would have been called a driving-assist tech and not auto-pilot.
intrasight 9y ago
I don't buy it either, and the airline business has lots of history to show that it is bunk. Autopilot must either be better than humans or not be present. I'm sure the car autopilot engineers have learned much from airplanes. And I'm pretty sure that Tesla management has overrode the engineers concerns because they'd rather move fast and break things.
theluketaylor 9y ago
>I don't even enable cruise control because taking the accelerator input away from me is enough to cause my mind to wander You're not alone. I find the act of modulating my speed is what keeps me focused on the task of driving safely. Steering alone isn't enough; I can stay in my lane without tracking the vehicles around me or fully comprehending road conditions. Until a Level 5 autonomous car is ready to drive me point A to point B while I watch a movie I will remain firmly in command of the vehicle.
- dbasedweeb 9y ago
  The problem as always with driving is that you can be as attentive, sober, cautious as humanly possible... while the guy who jumps the median into your windscreen may not be. We need to be more concerned and proactive about stopping this running experiment with half-asses automation in which we all unwillingly participate. I want lvl 5 automation just like anyone, but I don’t believe it’s anywhere close, and I’m not interested in being part of a Tesla or Uber’s attempt to be even richer. Public roads are not laboratories. It’s not just Tesla owners who are participating in this, it’s everyone on the road with them.
deleted 9y ago
[deleted]
kolpa 9y ago
The car should play "I spy" with you to keep you attentive to the road. You win if you spy a concrete barrier that the car doesn't.
unclebucknasty 9y ago
>I don't believe that you can have a car engaged in auto-drive mode and remain attentive I've been saying this for a while and it's interesting to see more people evolve to this point of view. There was a time when this idea was unpopular here--owed mostly to people claiming that autonomous cars are still safer than humans, so the risks were acceptable. I think there are philosophical and moral reasons why this is not good enough, but that goes off-topic a bit. In any case, some automakers have now embraced the Level-5 only approach and I sincerely believe that goal will not be achieved until either: 1. We achieve AGI or 2. Our roads are inspected and standards are set to certify them for autonomous vehicles (e.g. lane marking requirements, temporary construction changes, etc.) Otherwise, I don't believe we can simply unleash autonomous vehicles on any road in any conditions and expect them to perform perfectly. I also believe it's impossible to test for every scenario. The recent dashcam videos have convinced me further of this [0]. The fact that there are "known-unknowns" in the absence of complete test-ability is one major reason that "better than humans" is not an ethical standard. We simply can't release vehicles to the open roads when we know there are any situations in which a human would outperform them in potentially life-saving ways. [0] https://reddit.com/r/teslamotors/comments/8a0jfh/autopilot_barrier_lust_201812/ https://reddit.com/r/teslamotors/comments/8a0jfh/autopilot_b...
PinkMilkshake 9y ago
I agree, I find the same of myself, and I beat myself up over any loss of concentration. The solution might be a system where the driver drives at a higher level of abstraction, but ultimately still drives. Driving should be like declarative programming. For example, the driver still moves the steering wheel left and right, but the car handles the turn. Or when the driver hits the breaks, which is now more of a on/off switch, the car handles the breaking. The role for the driver is to remain engaged, indicating their intention at every moment and for the car to work out the details. Edit: On second thought, that might end up being worse. I can think of situations where it might become ambiguous to the driver of what they are handling and what the car is handling. Maybe autopilot is all or nothing.
facorreia 9y ago
I agree, and I think the psychological effect will be stronger if the feature is called autopilot. It conveys the idea that the car can pilot itself.
kojeovo 9y ago
> I don't believe that you can have a car engaged in auto-drive mode and remain attentive I can't even drive an automatic transmission car without getting way too distracted at times
modzu 9y ago
^^^ thanks for saying this, it is precisely the crux of the issue
mygo 9y ago
I try to tell people that if you cannot take a nap behind the wheel of a self-driving car, then the automaker has failed to produce a self driving car. If you have to be attentive behind the wheel of a self driving car, then you might as well just steer it yourself.
sokoloff 9y ago
As a user of autopilot in aviation context, I do remain engaged and connected to the flight while the autopilot handles the routine course following and altitude hold/tracking responsibilities. I don’t find that particularly challenging and in fact, when the autopilot is INOP, flights are slightly more mentally fatiguing because you have no offload and complex arrivals are much more work, but in cruise, you have to be paying attention either way. It’s not a time to read the newspaper, autopilot or not.

cdblades 9y ago

This same thing can be said of auto-pilot in planes. Not saying it invalidates your argument, just that there are pretty wide-reaching consequences to that idea.

mrguyorama 9y ago
Which is why pilots are still in control of the plane while autopilot is active. Even while active, there is still a "pilot flying" and pilots are still responsible for scanning gauges and readouts and verifying the autopilot is doing what they expect. They do not just turn on autopilot and goof off
- netc 9y ago
  they are expected to remain in control of the plane, not necessarily they do: https://timesofindia.indiatimes.com/india/Pilots-sleep-as-flight-attendant-turns-off-autopilot-on-Bangkok-Delhi-flight/articleshow/19860300.cms https://timesofindia.indiatimes.com/india/Pilots-sleep-as-fl...
- Johnny555 9y ago
  Just like Tesla's auto pilot? Drivers are supposed to be "flying", scanning gauges and the road ahead to ensure the autopilot is doing what they expect... I think people that say that "autopilot" is a bad name for this feature don't really understand what an "autopilot" does.
  - mannykannot 9y ago
    An airplane under autopilot is not a second or two away from a fatal crash.
    - Johnny555 9y ago
      If the pilots are not flying, then it can be just a short time away from a crash. Like when the pilot is not paying attention and by the time the auto pilot can no longer fly, the pilot doesn't have enough situational awareness to take over. http://www.slate.com/blogs/the_eye/2015/06/25/air_france_flight_447_and_the_safety_paradox_of_airline_automation_on_99.html http://www.slate.com/blogs/the_eye/2015/06/25/air_france_fli...
      - mannykannot 9y ago
        That is very much an outlier, and if it were at all relevant to the issue it would further weaken your case, as these three pilots had several minutes to sort things out. Questioning the assumptions underlying the assumed safety of airplane autopilot use can only weaken the claim that Tesla's 'autopilot' is safe.
    - mulmen 9y ago
      The fatal impact may not be seconds away but the event that sets in motion the series of actions that results in that fatal impact may take only seconds.
      - mannykannot 9y ago
        The issue is how long between the problem first manifesting itself and a crash becoming inevitable. Note that even as short a time as ten seconds is an order of magnitude longer than one second. There are at most only a few rare corner cases in aviation where proper use of the autopilot could take the airplane within a minute of an irretrievable situation that would not have occurred under manual control.
- stale2002 9y ago
  Uhhhhh, I think you are misinformed. They aren't going to literally fall asleep, but much of the time pilots are reading a book and not directly paying attention in the same way that a driver is.
krrrh 9y ago
This comparison really only works if you mean engaging autopilot with a bunch of other planes flying in close formation and if clouds were made of steel and concrete. Most of the time that autopilot is engaged on a flight there is next to zero risk of a collision. Commercial pilots also get a lot of training specifically related to what autopilot is and isn’t appropriate for.
- laythea 9y ago
  That is a valid counterpoint in my opinion.
- nightski 9y ago
  Even as a private pilot you are taught to continuously scan your instruments and surroundings as you fly through the air with autopilot enabled. Flight training is much more extensive than driving tests (although still not too bad) and they really drive procedures into you.
skrause 9y ago
Planes don't suddenly crash into a mountain because you have looked away for two seconds, there is much more time to react in case the autopilot doesn't behave correctly.
- z3t4 9y ago
  once you see the mountain it will be too late. think of big airplannes as trains.
- ptaipale 9y ago
  They can very suddenly crash into another plane because you have looked away for two seconds. It has actually happened (though nowadays, technology has made it easier to avoid these accidents).
  - AnimalMuppet 9y ago
    Planes are supposed to maintain a 5 mile separation distance. You aren't going to break that down in two seconds. (But head on, with both planes traveling 600 MPH, you can do it in 15 seconds. But both pilots would have to be inattentive for that time.)
    - ptaipale 9y ago
      They are supposed to, but if flight control doesn't help them do it, pilots have only very little time to react to what they meet. This was demonstrated in the Hughes Airwest collision with an F-4 in 1971. Since then, air traffic control procedures have been improved to avoid these situations, but nowadays e.g. over the Baltic Sea, Russian military planes are routinely flying with their transponder turned off so that flight control does not know where they are. So, this risk is still there. https://en.wikipedia.org/wiki/Hughes_Airwest_Flight_706#Probable_cause_and_recommendations https://en.wikipedia.org/wiki/Hughes_Airwest_Flight_706#Prob...

fapjacks 9y ago

I have to preface my post to say that I think developing self-driving automobiles is so important that it's worth the implied cost of potentially tens of thousands of lives in order to perfect the technology, because that's what people do; make sacrifices to improve the world we live in so that future generations don't have to know the same problem. But I think you're right. I think the "best" way to move forward until we have perfected the technology is not something that drives for you, but something that will completely take over the millisecond the car detects that something terrible is about to happen. People will be engaged because they have to be engaged, to drive the car. The machine can still gather all the data and ship it off to HQ to improve itself (and compare its own decisions to those of the human driver, which IMO is infinitely more valuable). But if there's one thing the average person is terrible at, it's reacting quickly to terrible situations. You're absolutely right that people can't be trusted to remain actively engaged when something else is doing the driving. Great example with the cruise control, too.

rightbyte 9y ago
Thats kinda Volvo Cars approuch, word by word, as one of their RnD engineers explained it to me.
- southern_cross 9y ago
  IIRC, it was also Volvo who a few years back said that they would gladly take on any liability issues for their self-driving cars. Only to backtrack on that a short while later after having learned what liability laws in the U.S. actually look like, saying that they wouldn't take on such liability until the laws are changed to be more in their favor. So there's that ...
almost_usual 9y ago
> I have to preface my post to say that I think developing self-driving automobiles is so important that it's worth the implied cost of potentially tens of thousands of lives in order to perfect the technology, because that's what people do; make sacrifices to improve the world we live in so that future generations don't have to know the same problem. I generally agree with this philosophy but this is very optimistic, at least in the United States. This is a country where we can't even ban assault rifles let alone people from driving their own vehicles. You're going to see people drive their own vehicles for a very long time even if self driving technology is perfected.

iro3939 9y ago

I think our psychology has been manipulated such that we ignore 1 million deaths a year by humans and don’t at all take seriously halting car sales But it hasn’t happened to me! says a rando not dealing with the death of a loved on or financial difficulty Gotta ship!

hippich 9y ago

Quite a lot years ago I had to drive for 6 hours straight at night to get on time to place where I needed to be (air flight was not available back then for me) What I noticed that when I was following posted/safe speed limit, I was quickly losing focus, mind started wandering and eventually I felt I was falling asleep. I do not remember what made me to speed up, but once I was about 30% faster than posted speed limits, and once I reached part of the way where road was quite bad + a lot of road work was happening, I realized that I much more alert. As soon as I slowed down to posted speed limit speed I began drifting away again.. If anything, my anecdote confirms your theory - as soon as we perceive something safer, we pay way less attention. And Autopilot sounds like one of these safety things, which makes drivers less attentive and potentially missing dangerous situation, which otherwise would be caught by driver's mind. I wonder if there is a way to introduce autopilot help without actually giving sense of security to the driver. Granted Tesla would lose so precious marketing angle, but if their autopilot would work somewhat like variable power steering system on background without obvious taking over control of the car, in the long haul that would be more beneficial?

pbhjpbhj 9y ago
Assuming you were in a vehicle with ICE speed relates to vibration/noise which can, at the wrong pitch, cause drowsiness easily. This is why parents will take their children out in the car if a child is not sleeping well. I find rough motorway surfaces in my current vehicle induce heavy drowsiness at motorway speed limits (slight reduced at marginally higher speeds when the pitch is higher).
- hippich 9y ago
  This is interesting idea, never thought about it! thanks!

chaostheory 9y ago

> This needs far more discussion. I just don't buy it. I don't believe that you can have a car engaged in auto-drive mode and remain attentive. I think our psychology won't allow it. Does anyone know of psychology studies that measure human reaction time and skill when sometime like autopilot is engaged most of the time? I remember taking part in a similar study at Georgia Tech that involved firing at a target using a joystick. It was also simultaneously a contest because only the top scorer would get prize money. The study was conducted in two parts. In the 1st phase, the system had autotargeting engaged. All subjects had to do was press a button when the reticle was on the target in order to score. In the 2nd phase, which was a surprise, autotargetting was turned off. I won the contest and my score was miles ahead of anyone. I can't fully confirm it but I feel this happened because I was still actively aiming for the target even when autotargetting was active.

Animats 9y ago
Does anyone know of psychology studies that measure human reaction time and skill when sometime like autopilot is engaged most of the time? Yes. That's been much studied in the aviation community.[1] NASA has the Multi-Attribute Test Battery to explicitly study this.[2] It runs on Windows with a joystick, and is available from NASA. The person being tested has several tasks, one of which is simply to keep a marker on target with the joystick as the marker drifts. This simulates the most basic flying task - flying straight and level. This task can be put on "autopilot", and when the marker drifts, the "autopilot" will simulate moving the joystick to correct the position. But sometimes the "autopilot" fails, and the marker starts drifting. The person being tested is supposed to notice this and take over. How long that takes is measured. That's exactly the situation which applies with Tesla's "autopilot". There are many studies using MATB. See the references. This is well explored territory in aviation. [1] http://web.ics.purdue.edu/~archerj/Course%20Deliverables/IE%20590%20(AAE%20590)%20-%20Effects%20of%20Automation%20in%20the%20Aircraft%20Cockpit%20Environment.pdf http://web.ics.purdue.edu/~archerj/Course%20Deliverables/IE%... [2] https://matb.larc.nasa.gov/software/ https://matb.larc.nasa.gov/software/
- chaostheory 9y ago
  This sounds identical to the study I participated in except it was conducted by Georgia Tech and not NASA.

cdancette 9y ago

For me it's the opposite. I feel like with cruise control on, I'm much more attentive to the road and the surroundings and feel much safer. It really annoys me when I have to constantly look at the speed to avoid getting above the limit and it is very tiring for long drives.

greglindahl 9y ago
I totally agree with this sentiment -- the only reason I drive a safe speed is that I use cruise control constantly. Then I don't have to think about speed, and I can focus on everything else.
kolpa 9y ago
You need a speed limiter, not necessarily cruise control.
- dhimes 9y ago
  Cruise control works fine for this.
- greglindahl 9y ago
  A speed limiter involves me having to continue to press on a pedal. That's quite different from cruise control.
flashmob 9y ago
Noticed this also, no need to monitor and adjust the speed which is a mundane task (in cruise control traffic conditions). Eyes can be on the road instead. This is similar to the problem for pilots, who can be distracted by mundane tasks due the complexity of controls in modern aircraft. If these tasks are removed, the pilot can focus on what's more important. According to NASA " For the most part, crews handle concurrent task demands efficiently, yet crew preoccupation with one task to the detriment of other tasks is one of the more common forms of error in the cockpit." https://asrs.arc.nasa.gov/publications/directline/dl10_distract.htm https://asrs.arc.nasa.gov/publications/directline/dl10_distr...
9y ago

cyberferret 9y ago

I agree with the psychology aspect of driving. I've seen it mentioned many times that a large majority of auto accidents occur a few minutes from the driver's home, and usually on their way home. Apparently, being close to their neighbourhood and in familiar surrounds, the driver's attention tends to wane as they get distracted with other things that they have to do when they get to their house. Racing drivers have also reported that when they are not driving at 100%, they are more prone to make mistakes or crash. Most famously, Ayrton Senna's infamous crash at Monaco when he was leading the field by a LONG way. When he was asked why he crashed at a fairly innocuous slow corner, he said that his engineer had asked him over the radio to 'take it easy' as there was no chance he would be challenged for 1st place before the finish line, so he relaxed a fraction and started thinking about the victory celebrations. And crashed.

intended 9y ago
I thought the point was that most car related jaunts are short ones.
kazinator 9y ago
Accidents take place close to home because driving takes place close to home and accidents take place where driving takes place. Basically this: https://xkcd.com/1138/ https://xkcd.com/1138/

kafquaesque 9y ago

If a person uses a device while using autopilot, (which seems highly likely—not sure in thi instance) wouldn’t it be advisable to have the alerts come to them directly on whichever device they are using? The alert breaks them out of whatever task they are focusing on. If the alert is coming from the car I can see how a lot of us could ignore it.

ehnto 9y ago
What a world. Where we can't even take enough responsibility to be present enough to hear the "You are about to die" bell chime. Imagine the possible breaking components in that chain too. Bluetooth can fail, satellite can fail, cell can fail, WiFi can fail, a USB cable can fail, there isn't a single piece of connectivity technology that would make me confident enough to delegate alerts to another device. There is also an inherent failure of alarms in general in that even very loud ones can be ignored if they give false positives even once or twice. There is a body of study trying to address it. Some of the most fatal industrial accidents occured because alarms were either ignored or even fully switched off. We aren't good with alarms. I think the meat of it though is that unless Auto-pilot works perfectly then you can't leave it alone. And if you can't leave it alone then what's the point? The sell for autonomous cars isn't that people are just so darn tired of turning the steering wheel that they would really rather not. It's that we could potentially be more productive if we could shift our full attention to work/study/relaxation while commuting.
- kafquaesque 9y ago
  It seems like we are in an “in-between” state where we are using humans to assuage the fears of people that aren’t sure if neural networks can drive better than humans. The goal is to eventually focus on something else. If it’s just about making driving safer I would think it’s more of an incremental innovation step versus the breakthrough concept of being able to do something else while being commuted completely by a neural-network driven vehicle. The bridge to get to this breakthrough hopefully isn’t hacked apart by naysayers. Every death should be met with empathy and a desire to strengthen this bridge and quicken the speed of crossing it.

hyperbole 9y ago

I don't buy this either nor should we it's not how the feature is marketed. "The system is designed to be able to conduct short and long distance trips with no action required by the person in the driver’s seat." The result of this statement and the functionality that matches it is it creates a re-enforced false sense of security. Does it matter whether the driver of the model X whose auto pilot drove straight into a center divider had his hands on the wheel if the outcome of applying autopilot is drivers focus less on the road? What is the point of two drivers one machine one human? You cannot compare car auto pilot to airplane they're not even in the same league. How often does a center divider just pop up at 20k ft? Usually machinery either augments human capabilities by enhancing them, or entirely replaces them. This union caused by both driver and car piloting the vehicle has no point especially when it's imperfect. I'm not opposed to Tesla's sale of such functionality, sell whatever you want, but I am opposed to the marketing material selling this in a way that contradicts the legal language required to protect Tesla... There's risks in everything you do, but don't market a car as having the hardware to do 2x your customers driving capability and then have your legal material say: * btw don't take your hands off the steering wheel... especially when there's a several minute video showing exactly that. Tesla customers must have the ability to make informed choices in the risks they take.

jameslevy 9y ago
There is a solution for this - if the driver shows any type of pattern of not using the feature safely, disable the feature. Autopilot and comparable functionality from other vehicles should be considered privileges that can be revoked.
- hyperbole 9y ago
  Systems that are semi autonomous where there's some expectation of intervention work well in those scenarios, keep the car in lane markers on the highway, etc. Make sure the users hands are on the wheel but for fully autonomous even if your hands are on the wheel, how does it know your paying attention?
  - 9y ago

paganel 9y ago

> When driving, I find that I must be engaged and on long trips I don't even enable cruise control because taking the accelerator input away from me is enough to cause my mind to wander. Glad I'm not the only doing it. When driving on a highway I increase or decrease my car's speed by 10-15 km every 10 minutes or so, so that this variation can help me keep attentive about my surroundings.

Retric 9y ago

Your belief is meaningless, we have hard data that shows a net benefit to these systems. It's not a question of zero deaths, it's a question of reducing the number which means you need to look beyond individual events. Remember the ~90 people who died yesterday from a US car accident without making the news are far more important than a few individuals.

phkahler 9y ago
>> our belief is meaningless, we have hard data that shows a net benefit to these systems. No we don't. Tesla likes to compare their deaths per mile to the national average. The problem is that their autopilot is not fit to drive everywhere or in all conditions that go into that average. There is no data to support that autopilot is safer overall. It may not even be safer in highway conditions given that we've seen it broadside a semi and now deviate from the lane into a barrier - both in normal to good conditions.
- Retric 9y ago
  Specific failures are again meaningless. Computers don't fail the way people do, on the other hand people also regularly fall asleep at the wheel and do similar dumb things. And really, driving conditions are responsible for a relatively small percentage of vehicle fatalities. Most often it's people doing really dumb things like driving 100+ MPH. The only thing we actually know is these cars are safer on average than similar cars without these systems. That's not looking at how much they are used, just the existence of said safety system and likely relates to them being used when the drivers are extremely drunk or tired which are both extremely dangerous independent of weather conditions.
  - kalleboo 9y ago
    So how about we adopt much cheaper and simpler solutions like drowsiness detection (Volvos have these), automatic emergency braking (I think every brand has this as an option now), breathalizer locks, speed limiters etc? The US just mandated all new cars have backup cameras, but it seems like mandating AEB would make a bigger difference.

pavanred 9y ago

I suppose its perhaps how they marketed the feature. We have parking assist feature in many cars, there's a reason its not auto-park instead. If it really was a feature to help attentive drivers avoid accidents, it probably would have been called a driving-assist tech and not auto-pilot.

intrasight 9y ago

I don't buy it either, and the airline business has lots of history to show that it is bunk. Autopilot must either be better than humans or not be present. I'm sure the car autopilot engineers have learned much from airplanes. And I'm pretty sure that Tesla management has overrode the engineers concerns because they'd rather move fast and break things.

theluketaylor 9y ago

>I don't even enable cruise control because taking the accelerator input away from me is enough to cause my mind to wander You're not alone. I find the act of modulating my speed is what keeps me focused on the task of driving safely. Steering alone isn't enough; I can stay in my lane without tracking the vehicles around me or fully comprehending road conditions. Until a Level 5 autonomous car is ready to drive me point A to point B while I watch a movie I will remain firmly in command of the vehicle.

dbasedweeb 9y ago
The problem as always with driving is that you can be as attentive, sober, cautious as humanly possible... while the guy who jumps the median into your windscreen may not be. We need to be more concerned and proactive about stopping this running experiment with half-asses automation in which we all unwillingly participate. I want lvl 5 automation just like anyone, but I don’t believe it’s anywhere close, and I’m not interested in being part of a Tesla or Uber’s attempt to be even richer. Public roads are not laboratories. It’s not just Tesla owners who are participating in this, it’s everyone on the road with them.

deleted 9y ago

[deleted]

kolpa 9y ago

The car should play "I spy" with you to keep you attentive to the road. You win if you spy a concrete barrier that the car doesn't.

unclebucknasty 9y ago

>I don't believe that you can have a car engaged in auto-drive mode and remain attentive I've been saying this for a while and it's interesting to see more people evolve to this point of view. There was a time when this idea was unpopular here--owed mostly to people claiming that autonomous cars are still safer than humans, so the risks were acceptable. I think there are philosophical and moral reasons why this is not good enough, but that goes off-topic a bit. In any case, some automakers have now embraced the Level-5 only approach and I sincerely believe that goal will not be achieved until either: 1. We achieve AGI or 2. Our roads are inspected and standards are set to certify them for autonomous vehicles (e.g. lane marking requirements, temporary construction changes, etc.) Otherwise, I don't believe we can simply unleash autonomous vehicles on any road in any conditions and expect them to perform perfectly. I also believe it's impossible to test for every scenario. The recent dashcam videos have convinced me further of this [0]. The fact that there are "known-unknowns" in the absence of complete test-ability is one major reason that "better than humans" is not an ethical standard. We simply can't release vehicles to the open roads when we know there are any situations in which a human would outperform them in potentially life-saving ways. [0] https://reddit.com/r/teslamotors/comments/8a0jfh/autopilot_barrier_lust_201812/ https://reddit.com/r/teslamotors/comments/8a0jfh/autopilot_b...

PinkMilkshake 9y ago

I agree, I find the same of myself, and I beat myself up over any loss of concentration. The solution might be a system where the driver drives at a higher level of abstraction, but ultimately still drives. Driving should be like declarative programming. For example, the driver still moves the steering wheel left and right, but the car handles the turn. Or when the driver hits the breaks, which is now more of a on/off switch, the car handles the breaking. The role for the driver is to remain engaged, indicating their intention at every moment and for the car to work out the details. Edit: On second thought, that might end up being worse. I can think of situations where it might become ambiguous to the driver of what they are handling and what the car is handling. Maybe autopilot is all or nothing.

facorreia 9y ago

I agree, and I think the psychological effect will be stronger if the feature is called autopilot. It conveys the idea that the car can pilot itself.

kojeovo 9y ago

> I don't believe that you can have a car engaged in auto-drive mode and remain attentive I can't even drive an automatic transmission car without getting way too distracted at times

modzu 9y ago

^^^ thanks for saying this, it is precisely the crux of the issue

mygo 9y ago

I try to tell people that if you cannot take a nap behind the wheel of a self-driving car, then the automaker has failed to produce a self driving car. If you have to be attentive behind the wheel of a self driving car, then you might as well just steer it yourself.

sokoloff 9y ago

As a user of autopilot in aviation context, I do remain engaged and connected to the flight while the autopilot handles the routine course following and altitude hold/tracking responsibilities. I don’t find that particularly challenging and in fact, when the autopilot is INOP, flights are slightly more mentally fatiguing because you have no offload and complex arrivals are much more work, but in cruise, you have to be paying attention either way. It’s not a time to read the newspaper, autopilot or not.