9 ms·

Should cars be fully driverless? No, says an MIT engineer and historian

57 points by rajathagasthya 11y ago

quanticle 11y ago
The thing that worries me about semi-autonomous cars is that we might end up in the same place we've ended up with aircraft. Aircraft are sufficiently autonomous in nominal conditions that pilot skills atrophy, and then when the plane encounters adverse conditions, these de-skilled pilots simply do not respond correctly to the problem, crashing the plane. Specifically, I'm thinking of something like Air France Flight 447, where it turned out that the pilot and co-pilot had lost familiarity with how the Airbus A330 behaved while at cruising altitude. Similarly, I'm worried that auto-drive and smart cruise control systems will take care of highway driving in the normal case, but when things go wrong and the auto-driver disengages, the human at the wheel will have forgotten that a car traveling at 60mph behaves differently than one traveling at 25mph. With aircraft, we can mitigate this risk by forcing pilots to spend extra time in the simulator, keeing their feel for the aircraft from decaying too badly. I'm not sure what the solution is for drivers. Maybe we can force everyone to play a racing sim for a few hours every month ;)
- jessaustin 11y ago
  This danger will be avoided in automobiles by making human control impossible as soon as that makes any sense. There will be no steering wheel, the front seat will face backwards to facilitate conversation, and accident rates will plummet. The thing that robocars can do, which roboplanes cannot, is slow down or stop completely when faced with vexing situations.
  - hguant 11y ago
    Pulling over to the side of the road and putting your blinkers on is (almost) always an acceptable solution to an automotive error. Much harder to pull that off at 35,000 feet.
    - sandworm101 11y ago
      It's apples and oranges. Overshooting a turn at by a few seconds at 35,000 feet isn't the end of the world. Missing a turn in a car by even a quarter second means blood on the pavement.
      - geon 11y ago
        > Missing a turn in a car by even a quarter second means blood on the pavement. For a race driver perhaps. In traffic, you should keep a low enough speed and large enough distance, that 250 ms isn't a big deal.
    - Animats 11y ago
      If you have a problem at cruising altitude, there's usually time to deal with it. Transport planes can glide 100-200 miles from that altitude if they have to. Most aircraft accidents happen during takeoff, approach, or landing. See NTSB reports.[1] High-altitude accidents are mostly in-flight fires, including two events involving lithium batteries. There is, however, one commercial aircraft crash from problems at altitude (no passengers, ferry flight) worthy of a Darwin Award.[2] [1] http://www.ntsb.gov/investigations/AccidentReports/Pages/aviation.aspx http://www.ntsb.gov/investigations/AccidentReports/Pages/avi... [2] http://www.ntsb.gov/investigations/AccidentReports/Reports/AAR0701.pdf http://www.ntsb.gov/investigations/AccidentReports/Reports/A...
      - deleted 11y ago
        [deleted]
      - jessaustin 11y ago
        That second report is fascinating. One might hope that even amateur pilots would know power curve characteristics at high altitude, and the flight team's "gosh we're saving a lot of fuel" comments on the CVR inspire a sort of grim amusement, but I wonder whether the incident might actually have saved lives in the long run. If we're going to fly budget carriers like Pinnacle, those carriers are going to employ inexperienced flight crews and attempt to make up it up with training, simulation, and rules. This misadventure identified numerous shortfalls in those. I doubt anyone flying today's Endeavor Air ignores so many stick shakes, uses the same stall procedure at all altitudes, ignores airspeed, hasn't heard of "core lock", or lies so blatantly to ATC in potential emergency situations. I can understand why the report soft-pedaled the training gaps, because after all they didn't need to give the aircrew's survivors any more leverage in the inevitable lawsuit, but I'd expect training to improve anyway. Is it common practice for pilots at budget airlines to read accident reports like this? IANAPilot, but I'd like to be!
  - snowwrestler 11y ago
    So how is the vexing situation resolved? Either the onboard computer must be able to manage 100% of all situations, or the human driver must have an option to take control to resolve it themselves. You can't have a car that makes human control impossible, and then gets itself into vexing situations. You'll be stuck there forever.
    - bloaf 11y ago
      Is it impossible for a human to get a car permanently stuck? *edit: put another way, is it possible for a human driver to extricate a car from any situation?
      - snowwrestler 11y ago
        Well the parent refers only to certain situations--those in which the robocar gets vexed and chooses to slow down or stop. I would expect humans to be able to resolve those types of situations, since we rarely find humans sitting in perfectly working cars on the side of the road just because they are vexed.
        bloaf 11y ago
        Ah, I see that you are referring to a scenario in which the vexation and stopping are not both products of some external stimulus (e.g. [1]) or malfunction but something else altogether. For example, parents who pull their perfectly working car to the side of the road to discipline their vexatious children. It certainly seems possible that a self-driving car might occasionally get itself into a situation requiring a reboot, but that seems like an awfully low price to pay for the convenience of autonomous driving. [1] http://extras.mnginteractive.com/live/media/site36/2007/0811/20070811_055248_op12bridge.jpg http://extras.mnginteractive.com/live/media/site36/2007/0811...
    - jessaustin 11y ago
      For nearly every mistake that human drivers make, a good way to prevent it is to slow down. Not every situation must be navigated at maximum speed. Robocars will be better drivers than humans, but slower will still be safer. I'm not talking about cars parked in the lane indefinitely because they're confused by some dirt obscuring the center line. If that had been a thing the Google cars would already have fixed it.
  - sandworm101 11y ago
    Why assume that accident rates will drop in the all-autodrive scenario? I haven't seen any such testing. I have seen many miles of google cars in amongst human-driven cars. I have yet to see any tests involving thousands of cars from a variety of manufacturers (and ages) all trying to drive together. To assume that the robots will cooperate perfectly, negating the needs for safety features ... wasn't there a movie about that? Big lizards in an autonomously-run park?
- afarrell 11y ago
  Silly suggestion: have the copilot pilot spend the flight playing crash-recovery scenarios that reset every `random.randint(30,90)` minutes. When there is an actual emergency, plug him in to the actual avionics. It would be kinda like [work of fiction you probably should have read by now].
  - lfowles 11y ago
    Closest that comes to mind is Ender's Game. Ender doing simulations that abruptly increase in difficulty.
  - sandworm101 11y ago
    Pilots might look like they are sitting on their asses up there, but they aren't. They are constantly evaluating situations. Think of them like a bodyguard. They might look like they are just standing around doing nothing, but in fact they are constantly scouting for that one-in-a-million scenario. You don't want them heads-down playing games all day.
- jfoutz 11y ago
  I thought the pilots got confused because of https://en.wikipedia.org/wiki/Flight_control_modes#Alternate_law https://en.wikipedia.org/wiki/Flight_control_modes#Alternate... not, particularly, because of de-skilling. I'm going to use a stupid metaphor, that's perhaps ton totally apt. In that case it's kind of like reversing the gas and break pedals. 99.9% of the time, practically all your time behind the wheel the car behaves one way, but in this one special circumstance, the 'normal' emergency reaction becomes exactly the wrong thing to do. Hit the brake not the gas, but the pedals are backwards. Yeah, maybe drivers will deskill. cars will fully automate much faster than planes. google's little fleet has racked up a million miles. The 80% solutions will turn into 95% solutions in a few iterations, then 99.99% solutions in a few more, then a long tail of fixing edge cases as they are discovered.
  - quanticle 11y ago
    While the confusion over normal mode and alternate mode did play a role in the crash, the BEA report investigating the crash found the following six proximate causes [1]: - temporary inconsistency between the measured speeds, likely as a result of the obstruction of the pitot tubes by ice crystals, causing autopilot disconnection and reconfiguration to alternate law; - the crew made inappropriate control inputs that destabilized the flight path; - the crew failed to follow appropriate procedure for loss of displayed airspeed information; - the crew were late in identifying and correcting the deviation from the flight path; - the crew lacked understanding of the approach to stall; - the crew failed to recognize that the aircraft had stalled and consequently did not make inputs that would have made it possible to recover from the stall. Of those six, three can be attributed to skills atrophy: failure to follow procedure for loss of displayed airspeed information, lateness in identifying deviation from the flight path, and failure to recognize stall. These are all skills that pilots are taught when they're certified to fly a given aircraft. The fact that they didn't have them means that they forgot, or, in other words, they didn't have enough ongoing training to keep their skills fresh for emergencies. [1] https://en.wikipedia.org/wiki/Air_France_Flight_447#Final_report https://en.wikipedia.org/wiki/Air_France_Flight_447#Final_re...
    - 11y ago
- deleted 11y ago
  [deleted]
- mrspeaker 11y ago
  I wouldn't worry too much - a lot of people already die on the roads in fully non-autonomous cars. We only need fewer people to die and it's worth it.
  - tonyedgecombe 11y ago
    That's a rational approach, I'm not sure that many politicians think like that though.
    - brixon 11y ago
      They do, but they will not phrase it that way.
Zikes 11y ago
I must be missing something, because it seems like Mr. Mindell is saying cars shouldn't be fully driverless because they can't be. He cites numerous examples of other promised automation technologies that seem to have become a pipe dream over the past several decades. Except, Google's driverless cars have now logged over a million fully automated miles. It is a proven technology. These aren't exploratory machines going where no human has gone before, they're out there on the road doing what we've all been doing for a hundred years already. I get the impression this article is just a shill for his book.
- sandworm101 11y ago
  I would suggest that Google cars haven't logged any driverless miles, at least not on public roads. They have humans onboard with backup controls (parking brakes if nothing else). So they are not driverless just yet.
  - ekianjo 11y ago
    a human not touching the wheel in most situations is a human that is NOT in control.
    - sandworm101 11y ago
      Hands off the wheel does not mean not in control if you can put those hands back on the wheel instantly. Take the hands off the wheel, get in the back seat, and take a nap. That's what I would call a driverless car. The "hands not on wheel" argument has been made by many a defendant. Courts don't buy it anymore. If you are in the driver seat, inches from the controls, then you are responsible for the motion of the vehicle. Which is why you can be charged for drunk driving even though the car never moved. In the seat or at the controls = in control = not driverless.
      - nemothekid 11y ago
        This is an incredibly pedantic definition and has nothing to do with what Google is doing. Google's car has logged driverless hours - despite where a human can be sitting. Are you seriously implying that a human sitting in the driver seat is controlling the car without touching it? If its not driverless, how is the car navigating? Through psychic powers?
jessaustin 11y ago
The Apollo program is hardly as convincing an example as TFA seems to think. That was early days for computing power. Humans have landed scores of other craft on various other bodies in the solar system, and that landing has been fully automated in every other case. If a manned moon mission were planned today it would probably be completely automated.
- Animats 11y ago
  The first moon landings (Surveyor, Lunokhod) were robotic and were completely automated. The boost phase has been completely automated all the way back to the first manned space flights. After all, a monkey made the first flight. Other than lunar landing and some older docking operations, just about everything in space flight has been automated.
  - icebraining 11y ago
    Automated or remotely controlled?
  - chii 11y ago
    automated can mean 'pre-programmed' automated, instead of dynamic automation, where the actions required aren't known ahead of time, and the machine(s) have a split second to make a choice out of the hundreds of different options available.
zhanwei 11y ago
He is comparing situations where human decision makers are highly trained professionals such as astronauts and passenger plane pilot to daily driving situations where the average human decision maker is more likely to be an idiot(people switch off while driving) distracted by his smartphone.
- sandworm101 11y ago
  Astronauts and pilots also switch off. It's not a function of training, but boredom and/or distraction.
  - TeMPOraL 11y ago
    Which only strenghtens the argument for self-driving cars. Hell, we even make those highly-trained professional aviators and astronauts depend on auto-pilots because it's much safer this way.
    - sandworm101 11y ago
      So long as the self-driving car can cover every situation. What doesn't work is a system that expects the human to sit idle for hundreds of hours, then suddenly throws him/her the wheel and jumps ship. (Like the autodrive system in that simpsons episode where homer becomes a trucker.)
      - TeMPOraL 11y ago
        I agree. Self-driving system that expects humans to take over randomly every now and then when something unexpected happens is much more dangerous than 100% self-driving or no self-driving - it exposes only the worst aspects of human driving while removing the best.
        Lewton 11y ago
        That's not completely clear. It depends entirely on how often the human is expected to take over
ekianjo 11y ago
Erm... the main point he makes is like "oh, we could not do it before no matter how hard we tried, so let's give up and keep humans in charge" basically. I'm sure no one believe we would send rockets into space if you look back long enough in time, yet we did it. Driverless cars are not a concept anymore, many prototypes are very close to being fully operational - it would be very disappointing to stop at this stage.
- Turing_Machine 11y ago
  "I'm sure no one believe we would send rockets into space if you look back long enough in time" Not even all that long ago, actually. http://www.popsci.com/military-aviation-amp-space/article/2009-07/new-york-times-nasa-youre-right-rockets-do-work-space http://www.popsci.com/military-aviation-amp-space/article/20...
namuol 11y ago
> “That’s just proven to be a loser of an approach in a lot of other domains,” Mindell says. “I’m not arguing this from first principles. There are 40 years’ worth of examples.” Yeah, and I have several millennia worth of examples suggesting that organized society at large is a worthless endeavor...
andrewray 11y ago
Cars seem fundamentally different than his other examples: Spacecraft, underwater vehicles, and airplanes. The importance difference is cars have a clear, visual grid of where they can travel and where they cannot travel. Painted lines on the road and navigational signs provide a construct of the environment around them. Maybe the closest comparison is subway / metro rails. There are already many fully automated subway systems: https://en.wikipedia.org/wiki/List_of_automated_urban_metro_subway_systems#Grade_of_Automation_4_Systems https://en.wikipedia.org/wiki/List_of_automated_urban_metro_...
- zhanwei 11y ago
  There are still some elements of uncertainty in car driving. For e.g. driverless cars are confused by cyclist doing track stand[1]. Hopefully with enough data driverless car could overcome these difficulties. [1] http://road.cc/content/news/162468-cyclist-doing-trackstand-leaves-googles-self-driving-car-confused http://road.cc/content/news/162468-cyclist-doing-trackstand-...
  - icc97 11y ago
    Interesting story. There's a later article [1] that describes the problems that the Google car has with assertiveness at junctions. [1]: http://www.roboticstrends.com/article/a_cyclists_encounter_with_an_indecisive_google_self_driving_car http://www.roboticstrends.com/article/a_cyclists_encounter_w...
- snowwrestler 11y ago
  The hard part of self-driving cars is not staying within the painted lines of the road, the hard part is avoiding other valuable things like cars, trucks, bikes, people, wildlife, etc. There is no reliable visual grid for those things. Subways handle this problem by being given dedicated underground tunnels which exclude all those things. And even subways hit people sometimes when they fall onto the tracks.
- Tideflat 11y ago
  In Northern climes, the painted lines on the road are often covered by snow.
arikrak 11y ago
I think it may take decades before cars are 100% autonomous in all situations. But if the AI can handle almost everything, it could take over and let the humans read a book, and alert them in the rare case their input is needed. Fleets of autonomous taxis and trucks could have remote controlled operators monitoring them.
thadjo 11y ago
Mindel isn't saying full automation can't or shouldn't happen for cars. His point is empirical and cautionary: usually when we get excited about automating X, it turns out that X is better with some human supervision. It's not really a contentious claim. I think many proponents of self-driving cars would be ok with some human input – steering wheels, changing routes, feedback on driving performance, etc
- userbinator 11y ago
  I think many proponents of self-driving cars would be ok with some human input – steering wheels, changing routes, feedback on driving performance, etc AKA "cruise control"...?
aclissold 11y ago
On the topic of boredom and attention lapses while driving, I wonder if it would actually be made safer by being made more difficult, in contrast to the never-ending search for ease of use. Something that's difficult, mentally stimulating, and requires actual skill like a video game or musical instrument. Would there be more or less mistakes (collisions) if people were forced to be engaged 100% of the time in order to operate the vehicle? I'm not saying it's a good idea, because only a subset of people would be capable of it, but it's interesting to think about!
- sgibat 11y ago
  this is why many people prefer driving a manual.
  - burkemw3 11y ago
    Including me, so I tried to find some statistics to see if a manual was enough to focus on. I didn't find much. I found one Swedish study that looked at automatic and manual transmissions affect on young and old drivers' ability to turn left across traffic [1]. It found a manual had little impact on the young driver, and older drivers performed worse. I'm not sure turning left across traffic is a large enough sample of driving for me to be satisfied. [1] http://acrs.org.au/files/arsrpe/Why%20drive%20manual%20-%20automatic%20transmission%20improves%20driving.pdf http://acrs.org.au/files/arsrpe/Why%20drive%20manual%20-%20a...
- TeMPOraL 11y ago
  > Would there be more or less mistakes (collisions) if people were forced to be engaged 100% of the time in order to operate the vehicle? I think there would be less mistakes, but the problem is, you can't sustain the required level of engagement for long. 100% concentration for 15 minutes? Sure. 30? Yes. One hour? Maybe. Three hours? Probably not. Though, come to think of it, one hour would probably be enough to cover like 95% cases of commuting.
- agumonkey 11y ago
  Taken to some extreme, people realized that safer roads, no trees on the sides, aren't that safer, they just make people lazier and keep crashing even when trees have been removed, faster.
burkemw3 11y ago
In most of the given examples, humans have little control of the environment. We are, in fact, trying to use autonomous vehicles to explore. In the case of cars, humans can change the physical environment to meet the needs of the car. Additionally, the car may be able to avoid a situation that it knows it doesn't handle well (e.g. avoid complicated intersections). In the one case of an environment where we have some controls (air travel), it is more difficult to "pull over" as compared to a car (as other commenters have noted).
paulsutter 11y ago
While the article doesn't actually use the word "never", neither does it hedge and say "anytime soon". The central logic seems to be that humanity is starting from scratch building self driving cars, versus all the years of experience decades ago trying to automate spacecraft and submersibles. Oddly, I reach entirely the opposite conclusion. The article reminds us that even decades ago, with rudimentary computers, engineers developed systems that landed men on the moon and returned them to earth. This article has made it even easier for me to imagine a car taking a fully autonomous trip to a McDonald's drive through, given the state of deep/machine learning, low cost computers a million times faster, and big advances in sensors, Yes there will be a person supervising the car for a while, of course that's true. But it's only a matter of time before our children are surprised that humans were allowed to drive cars, causing millions of deaths.
- agumonkey 11y ago
  Landing on the moon is to physics what car traffic is to chaos theory.
  - minority-one 11y ago
    And it's weird how people bring up the idea of an elevator to the moon more often than they bring up putting cars to run on rails. They could be 100% self-driving that way, and accidents would probably be nil.
  - paulsutter 11y ago
    True, but cars can already navigate traffic autonomously. And there is much more to landing men on the moon than physics. Check out Tesla's live shipping code: http://jalopnik.com/teslas-autopilot-system-is-awesome-and-creepy-and-a-sig-1736573089 http://jalopnik.com/teslas-autopilot-system-is-awesome-and-c... Are you predicting that cars will never be fully autonomous?
    - agumonkey 11y ago
      As a website commenter I'm not sure I should predict anything. To explain a bit more my previous message, as complex as going to the moon was, it seems a problem scientists knew how to define. One vehicle, 2 planets, the system is describable. Traffic with lots of agents potentially interacting with every other agent, hence my chaos analogy, lots and lots of variable, ... is blurry. There's successful research to describe multi agents collision avoidance but I've never seen it used on the field yet. ps: reminds me airplanes vs car accident rates. Even if flying is inherently complex in itself, there's almost no traffic and almost no obstacle in air. Nobody would crash a car alone in the desert.
sklogic 11y ago
His 40 years of experience obviously did not include the only recently available GPUs and ASIC deep neural networks which are at the heart of the current self-driving robotics research. There is a game changer, and any previous experience is irrelevant.
spullara 11y ago
They are going to be interviewing him in a driverless car in 10 years and asking where he got it wrong.
pgodzin 11y ago
I think the difference between self-driving cars and Apollo program/subs examples is that the cars can be tested much more thoroughly. No matter how good your simulators are for space conditions, you're never going to really know how it's going to work in actual conditions. Self-driving cars can and are logging millions of miles in a variety of conditions, and every common and almost every remote case will be dealt with before launched to the general public.
highfreq 11y ago
40 years of history showing failure, of course for most of that time they have had between a millionth and half the computing power we have today, and most of those years had now where near the capability of image sensors and other sensors. So how much weight should we really give history on something like this?
- simonh 11y ago
  I don't think they're arguing against automation, just that completely driverless vehicles won't be the optimum path. I think there will be some applications for which fully driverless vehicles will be fine, such as trucks transporting goods between loading bays and taxis collecting and dropping off at street addresses, and many others where some level of driver control will be useful. Take their example of submarines. Fully autonomous submarines are completely possible, but what they point out is that some level of human control is more useful. They don't say why, but my guess is that this is because if the sub comes across something unusual that merits further investigation, humans are better at making that judgement call than a robot would. The same principle applies to the Mars rovers. But it also applies to plenty of routine uses of vehicles. Most of the time I'm driving from my house to a car park or street parking space and such journeys are probably completely automatable. But sometimes I arrive at e.g. a country fair and the parking is in a field and I know I want to park over there in a place which will soon be shaded by that tree, and not in a different place where it looks a bit muddy. At times like that I would want much more direct control over the vehicle and an automated system would not be able to make such decisions for me. So while the article does a poor job saying so, I think the point is that 100% full driverless automation isn't the best answer in every situation. It might be a fine solution for many purposes, but not all. They're arguing against a (perhaps imagined and possibly straw man) maximalist automation position such as 'in 20 years time no cars will have manual controls'.
meesterdude 11y ago
Here's the thing: we can do anything we want. And there is a huge demand for fully driverless cars that nobody is going to be able to stop, despite whatever sound and reasoned advice they may have. It's coming, like it or not.
rebootthesystem 11y ago
> “That’s just proven to be a loser of an approach in a lot of other domains,” Mindell says. “I’m not arguing this from first principles. There are 40 years’ worth of examples.” Well, that's the problem. Everything the author says is supported by the last N decades in engineering. That argument is so flawed it is astounding that it is coming out of someone from MIT. In looking at his bio it seems he has dedicated quite a bit of time to teaching and researching the history of engineering. It might just be the case that he isn't fully up to date on topics relevant to automation. Does he have an up-to-date CS/AI background? How about computing or chip design? Saying that automation has failed to deliver in the past N decades is like the Wright Brothers saying "let's not bother because powered flight has failed to be developed in the past N decades". That is not how engineering works. Today hundreds of millions of people have phones in their pockets that are orders of magnitude more powerful than the computers decades ago. We have better, smaller, cheaper sensors of all kinds. We have accumulated knowledge in relevant CS and AI fields. Today we have cars that can actually drive themselves to various degrees under their own computing power (without having to communicate with a supercomputer in a building). And this is just the beginning. Fully auto-drive cars will happen. Of course, they will retain the ability to be human driven as well for some time, probably decades. There are many reason for which that makes sense. I am fairly confident we will get to the point where the average self-driver will be safer than the average human driver. All you have to do is point at a teenager behind the wheel. Commutes will be shorter (due to driving efficiently using data) and safer and, with electric cars, healthier (you won't have to breathe the crap coming out of all those exhaust pipes). Besides, think of the wasted potential of millions of people staring at a road for hours rather than doing something more useful. If you commute one hour per day you'll get back 250 hours per year.

The thing that worries me about semi-autonomous cars is that we might end up in the same place we've ended up with aircraft. Aircraft are sufficiently autonomous in nominal conditions that pilot skills atrophy, and then when the plane encounters adverse conditions, these de-skilled pilots simply do not respond correctly to the problem, crashing the plane. Specifically, I'm thinking of something like Air France Flight 447, where it turned out that the pilot and co-pilot had lost familiarity with how the Airbus A330 behaved while at cruising altitude. Similarly, I'm worried that auto-drive and smart cruise control systems will take care of highway driving in the normal case, but when things go wrong and the auto-driver disengages, the human at the wheel will have forgotten that a car traveling at 60mph behaves differently than one traveling at 25mph. With aircraft, we can mitigate this risk by forcing pilots to spend extra time in the simulator, keeing their feel for the aircraft from decaying too badly. I'm not sure what the solution is for drivers. Maybe we can force everyone to play a racing sim for a few hours every month ;)

jessaustin 11y ago
This danger will be avoided in automobiles by making human control impossible as soon as that makes any sense. There will be no steering wheel, the front seat will face backwards to facilitate conversation, and accident rates will plummet. The thing that robocars can do, which roboplanes cannot, is slow down or stop completely when faced with vexing situations.
- hguant 11y ago
  Pulling over to the side of the road and putting your blinkers on is (almost) always an acceptable solution to an automotive error. Much harder to pull that off at 35,000 feet.
  - sandworm101 11y ago
    It's apples and oranges. Overshooting a turn at by a few seconds at 35,000 feet isn't the end of the world. Missing a turn in a car by even a quarter second means blood on the pavement.
    - geon 11y ago
      > Missing a turn in a car by even a quarter second means blood on the pavement. For a race driver perhaps. In traffic, you should keep a low enough speed and large enough distance, that 250 ms isn't a big deal.
  - Animats 11y ago
    If you have a problem at cruising altitude, there's usually time to deal with it. Transport planes can glide 100-200 miles from that altitude if they have to. Most aircraft accidents happen during takeoff, approach, or landing. See NTSB reports.[1] High-altitude accidents are mostly in-flight fires, including two events involving lithium batteries. There is, however, one commercial aircraft crash from problems at altitude (no passengers, ferry flight) worthy of a Darwin Award.[2] [1] http://www.ntsb.gov/investigations/AccidentReports/Pages/aviation.aspx http://www.ntsb.gov/investigations/AccidentReports/Pages/avi... [2] http://www.ntsb.gov/investigations/AccidentReports/Reports/AAR0701.pdf http://www.ntsb.gov/investigations/AccidentReports/Reports/A...
    - deleted 11y ago
      [deleted]
    - jessaustin 11y ago
      That second report is fascinating. One might hope that even amateur pilots would know power curve characteristics at high altitude, and the flight team's "gosh we're saving a lot of fuel" comments on the CVR inspire a sort of grim amusement, but I wonder whether the incident might actually have saved lives in the long run. If we're going to fly budget carriers like Pinnacle, those carriers are going to employ inexperienced flight crews and attempt to make up it up with training, simulation, and rules. This misadventure identified numerous shortfalls in those. I doubt anyone flying today's Endeavor Air ignores so many stick shakes, uses the same stall procedure at all altitudes, ignores airspeed, hasn't heard of "core lock", or lies so blatantly to ATC in potential emergency situations. I can understand why the report soft-pedaled the training gaps, because after all they didn't need to give the aircrew's survivors any more leverage in the inevitable lawsuit, but I'd expect training to improve anyway. Is it common practice for pilots at budget airlines to read accident reports like this? IANAPilot, but I'd like to be!
- snowwrestler 11y ago
  So how is the vexing situation resolved? Either the onboard computer must be able to manage 100% of all situations, or the human driver must have an option to take control to resolve it themselves. You can't have a car that makes human control impossible, and then gets itself into vexing situations. You'll be stuck there forever.
  - bloaf 11y ago
    Is it impossible for a human to get a car permanently stuck? *edit: put another way, is it possible for a human driver to extricate a car from any situation?
    - snowwrestler 11y ago
      Well the parent refers only to certain situations--those in which the robocar gets vexed and chooses to slow down or stop. I would expect humans to be able to resolve those types of situations, since we rarely find humans sitting in perfectly working cars on the side of the road just because they are vexed.
      - bloaf 11y ago
        Ah, I see that you are referring to a scenario in which the vexation and stopping are not both products of some external stimulus (e.g. [1]) or malfunction but something else altogether. For example, parents who pull their perfectly working car to the side of the road to discipline their vexatious children. It certainly seems possible that a self-driving car might occasionally get itself into a situation requiring a reboot, but that seems like an awfully low price to pay for the convenience of autonomous driving. [1] http://extras.mnginteractive.com/live/media/site36/2007/0811/20070811_055248_op12bridge.jpg http://extras.mnginteractive.com/live/media/site36/2007/0811...
  - jessaustin 11y ago
    For nearly every mistake that human drivers make, a good way to prevent it is to slow down. Not every situation must be navigated at maximum speed. Robocars will be better drivers than humans, but slower will still be safer. I'm not talking about cars parked in the lane indefinitely because they're confused by some dirt obscuring the center line. If that had been a thing the Google cars would already have fixed it.
- sandworm101 11y ago
  Why assume that accident rates will drop in the all-autodrive scenario? I haven't seen any such testing. I have seen many miles of google cars in amongst human-driven cars. I have yet to see any tests involving thousands of cars from a variety of manufacturers (and ages) all trying to drive together. To assume that the robots will cooperate perfectly, negating the needs for safety features ... wasn't there a movie about that? Big lizards in an autonomously-run park?
afarrell 11y ago
Silly suggestion: have the copilot pilot spend the flight playing crash-recovery scenarios that reset every `random.randint(30,90)` minutes. When there is an actual emergency, plug him in to the actual avionics. It would be kinda like [work of fiction you probably should have read by now].
- lfowles 11y ago
  Closest that comes to mind is Ender's Game. Ender doing simulations that abruptly increase in difficulty.
- sandworm101 11y ago
  Pilots might look like they are sitting on their asses up there, but they aren't. They are constantly evaluating situations. Think of them like a bodyguard. They might look like they are just standing around doing nothing, but in fact they are constantly scouting for that one-in-a-million scenario. You don't want them heads-down playing games all day.
jfoutz 11y ago
I thought the pilots got confused because of https://en.wikipedia.org/wiki/Flight_control_modes#Alternate_law https://en.wikipedia.org/wiki/Flight_control_modes#Alternate... not, particularly, because of de-skilling. I'm going to use a stupid metaphor, that's perhaps ton totally apt. In that case it's kind of like reversing the gas and break pedals. 99.9% of the time, practically all your time behind the wheel the car behaves one way, but in this one special circumstance, the 'normal' emergency reaction becomes exactly the wrong thing to do. Hit the brake not the gas, but the pedals are backwards. Yeah, maybe drivers will deskill. cars will fully automate much faster than planes. google's little fleet has racked up a million miles. The 80% solutions will turn into 95% solutions in a few iterations, then 99.99% solutions in a few more, then a long tail of fixing edge cases as they are discovered.
- quanticle 11y ago
  While the confusion over normal mode and alternate mode did play a role in the crash, the BEA report investigating the crash found the following six proximate causes [1]: - temporary inconsistency between the measured speeds, likely as a result of the obstruction of the pitot tubes by ice crystals, causing autopilot disconnection and reconfiguration to alternate law; - the crew made inappropriate control inputs that destabilized the flight path; - the crew failed to follow appropriate procedure for loss of displayed airspeed information; - the crew were late in identifying and correcting the deviation from the flight path; - the crew lacked understanding of the approach to stall; - the crew failed to recognize that the aircraft had stalled and consequently did not make inputs that would have made it possible to recover from the stall. Of those six, three can be attributed to skills atrophy: failure to follow procedure for loss of displayed airspeed information, lateness in identifying deviation from the flight path, and failure to recognize stall. These are all skills that pilots are taught when they're certified to fly a given aircraft. The fact that they didn't have them means that they forgot, or, in other words, they didn't have enough ongoing training to keep their skills fresh for emergencies. [1] https://en.wikipedia.org/wiki/Air_France_Flight_447#Final_report https://en.wikipedia.org/wiki/Air_France_Flight_447#Final_re...
  - 11y ago
deleted 11y ago
[deleted]
mrspeaker 11y ago
I wouldn't worry too much - a lot of people already die on the roads in fully non-autonomous cars. We only need fewer people to die and it's worth it.
- tonyedgecombe 11y ago
  That's a rational approach, I'm not sure that many politicians think like that though.
  - brixon 11y ago
    They do, but they will not phrase it that way.

Zikes 11y ago

I must be missing something, because it seems like Mr. Mindell is saying cars shouldn't be fully driverless because they can't be. He cites numerous examples of other promised automation technologies that seem to have become a pipe dream over the past several decades. Except, Google's driverless cars have now logged over a million fully automated miles. It is a proven technology. These aren't exploratory machines going where no human has gone before, they're out there on the road doing what we've all been doing for a hundred years already. I get the impression this article is just a shill for his book.

sandworm101 11y ago
I would suggest that Google cars haven't logged any driverless miles, at least not on public roads. They have humans onboard with backup controls (parking brakes if nothing else). So they are not driverless just yet.
- ekianjo 11y ago
  a human not touching the wheel in most situations is a human that is NOT in control.
  - sandworm101 11y ago
    Hands off the wheel does not mean not in control if you can put those hands back on the wheel instantly. Take the hands off the wheel, get in the back seat, and take a nap. That's what I would call a driverless car. The "hands not on wheel" argument has been made by many a defendant. Courts don't buy it anymore. If you are in the driver seat, inches from the controls, then you are responsible for the motion of the vehicle. Which is why you can be charged for drunk driving even though the car never moved. In the seat or at the controls = in control = not driverless.
    - nemothekid 11y ago
      This is an incredibly pedantic definition and has nothing to do with what Google is doing. Google's car has logged driverless hours - despite where a human can be sitting. Are you seriously implying that a human sitting in the driver seat is controlling the car without touching it? If its not driverless, how is the car navigating? Through psychic powers?

jessaustin 11y ago

The Apollo program is hardly as convincing an example as TFA seems to think. That was early days for computing power. Humans have landed scores of other craft on various other bodies in the solar system, and that landing has been fully automated in every other case. If a manned moon mission were planned today it would probably be completely automated.

Animats 11y ago
The first moon landings (Surveyor, Lunokhod) were robotic and were completely automated. The boost phase has been completely automated all the way back to the first manned space flights. After all, a monkey made the first flight. Other than lunar landing and some older docking operations, just about everything in space flight has been automated.
- icebraining 11y ago
  Automated or remotely controlled?
- chii 11y ago
  automated can mean 'pre-programmed' automated, instead of dynamic automation, where the actions required aren't known ahead of time, and the machine(s) have a split second to make a choice out of the hundreds of different options available.

zhanwei 11y ago

He is comparing situations where human decision makers are highly trained professionals such as astronauts and passenger plane pilot to daily driving situations where the average human decision maker is more likely to be an idiot(people switch off while driving) distracted by his smartphone.

sandworm101 11y ago
Astronauts and pilots also switch off. It's not a function of training, but boredom and/or distraction.
- TeMPOraL 11y ago
  Which only strenghtens the argument for self-driving cars. Hell, we even make those highly-trained professional aviators and astronauts depend on auto-pilots because it's much safer this way.
  - sandworm101 11y ago
    So long as the self-driving car can cover every situation. What doesn't work is a system that expects the human to sit idle for hundreds of hours, then suddenly throws him/her the wheel and jumps ship. (Like the autodrive system in that simpsons episode where homer becomes a trucker.)
    - TeMPOraL 11y ago
      I agree. Self-driving system that expects humans to take over randomly every now and then when something unexpected happens is much more dangerous than 100% self-driving or no self-driving - it exposes only the worst aspects of human driving while removing the best.
      - Lewton 11y ago
        That's not completely clear. It depends entirely on how often the human is expected to take over

ekianjo 11y ago

Erm... the main point he makes is like "oh, we could not do it before no matter how hard we tried, so let's give up and keep humans in charge" basically. I'm sure no one believe we would send rockets into space if you look back long enough in time, yet we did it. Driverless cars are not a concept anymore, many prototypes are very close to being fully operational - it would be very disappointing to stop at this stage.

Turing_Machine 11y ago
"I'm sure no one believe we would send rockets into space if you look back long enough in time" Not even all that long ago, actually. http://www.popsci.com/military-aviation-amp-space/article/2009-07/new-york-times-nasa-youre-right-rockets-do-work-space http://www.popsci.com/military-aviation-amp-space/article/20...

namuol 11y ago

> “That’s just proven to be a loser of an approach in a lot of other domains,” Mindell says. “I’m not arguing this from first principles. There are 40 years’ worth of examples.” Yeah, and I have several millennia worth of examples suggesting that organized society at large is a worthless endeavor...

andrewray 11y ago

Cars seem fundamentally different than his other examples: Spacecraft, underwater vehicles, and airplanes. The importance difference is cars have a clear, visual grid of where they can travel and where they cannot travel. Painted lines on the road and navigational signs provide a construct of the environment around them. Maybe the closest comparison is subway / metro rails. There are already many fully automated subway systems: https://en.wikipedia.org/wiki/List_of_automated_urban_metro_subway_systems#Grade_of_Automation_4_Systems https://en.wikipedia.org/wiki/List_of_automated_urban_metro_...

zhanwei 11y ago
There are still some elements of uncertainty in car driving. For e.g. driverless cars are confused by cyclist doing track stand[1]. Hopefully with enough data driverless car could overcome these difficulties. [1] http://road.cc/content/news/162468-cyclist-doing-trackstand-leaves-googles-self-driving-car-confused http://road.cc/content/news/162468-cyclist-doing-trackstand-...
- icc97 11y ago
  Interesting story. There's a later article [1] that describes the problems that the Google car has with assertiveness at junctions. [1]: http://www.roboticstrends.com/article/a_cyclists_encounter_with_an_indecisive_google_self_driving_car http://www.roboticstrends.com/article/a_cyclists_encounter_w...
snowwrestler 11y ago
The hard part of self-driving cars is not staying within the painted lines of the road, the hard part is avoiding other valuable things like cars, trucks, bikes, people, wildlife, etc. There is no reliable visual grid for those things. Subways handle this problem by being given dedicated underground tunnels which exclude all those things. And even subways hit people sometimes when they fall onto the tracks.
Tideflat 11y ago
In Northern climes, the painted lines on the road are often covered by snow.

arikrak 11y ago

I think it may take decades before cars are 100% autonomous in all situations. But if the AI can handle almost everything, it could take over and let the humans read a book, and alert them in the rare case their input is needed. Fleets of autonomous taxis and trucks could have remote controlled operators monitoring them.

thadjo 11y ago

Mindel isn't saying full automation can't or shouldn't happen for cars. His point is empirical and cautionary: usually when we get excited about automating X, it turns out that X is better with some human supervision. It's not really a contentious claim. I think many proponents of self-driving cars would be ok with some human input – steering wheels, changing routes, feedback on driving performance, etc

userbinator 11y ago
I think many proponents of self-driving cars would be ok with some human input – steering wheels, changing routes, feedback on driving performance, etc AKA "cruise control"...?

aclissold 11y ago

On the topic of boredom and attention lapses while driving, I wonder if it would actually be made safer by being made more difficult, in contrast to the never-ending search for ease of use. Something that's difficult, mentally stimulating, and requires actual skill like a video game or musical instrument. Would there be more or less mistakes (collisions) if people were forced to be engaged 100% of the time in order to operate the vehicle? I'm not saying it's a good idea, because only a subset of people would be capable of it, but it's interesting to think about!

sgibat 11y ago
this is why many people prefer driving a manual.
- burkemw3 11y ago
  Including me, so I tried to find some statistics to see if a manual was enough to focus on. I didn't find much. I found one Swedish study that looked at automatic and manual transmissions affect on young and old drivers' ability to turn left across traffic [1]. It found a manual had little impact on the young driver, and older drivers performed worse. I'm not sure turning left across traffic is a large enough sample of driving for me to be satisfied. [1] http://acrs.org.au/files/arsrpe/Why%20drive%20manual%20-%20automatic%20transmission%20improves%20driving.pdf http://acrs.org.au/files/arsrpe/Why%20drive%20manual%20-%20a...
TeMPOraL 11y ago
> Would there be more or less mistakes (collisions) if people were forced to be engaged 100% of the time in order to operate the vehicle? I think there would be less mistakes, but the problem is, you can't sustain the required level of engagement for long. 100% concentration for 15 minutes? Sure. 30? Yes. One hour? Maybe. Three hours? Probably not. Though, come to think of it, one hour would probably be enough to cover like 95% cases of commuting.
agumonkey 11y ago
Taken to some extreme, people realized that safer roads, no trees on the sides, aren't that safer, they just make people lazier and keep crashing even when trees have been removed, faster.

burkemw3 11y ago

In most of the given examples, humans have little control of the environment. We are, in fact, trying to use autonomous vehicles to explore. In the case of cars, humans can change the physical environment to meet the needs of the car. Additionally, the car may be able to avoid a situation that it knows it doesn't handle well (e.g. avoid complicated intersections). In the one case of an environment where we have some controls (air travel), it is more difficult to "pull over" as compared to a car (as other commenters have noted).

paulsutter 11y ago

While the article doesn't actually use the word "never", neither does it hedge and say "anytime soon". The central logic seems to be that humanity is starting from scratch building self driving cars, versus all the years of experience decades ago trying to automate spacecraft and submersibles. Oddly, I reach entirely the opposite conclusion. The article reminds us that even decades ago, with rudimentary computers, engineers developed systems that landed men on the moon and returned them to earth. This article has made it even easier for me to imagine a car taking a fully autonomous trip to a McDonald's drive through, given the state of deep/machine learning, low cost computers a million times faster, and big advances in sensors, Yes there will be a person supervising the car for a while, of course that's true. But it's only a matter of time before our children are surprised that humans were allowed to drive cars, causing millions of deaths.

agumonkey 11y ago
Landing on the moon is to physics what car traffic is to chaos theory.
- minority-one 11y ago
  And it's weird how people bring up the idea of an elevator to the moon more often than they bring up putting cars to run on rails. They could be 100% self-driving that way, and accidents would probably be nil.
- paulsutter 11y ago
  True, but cars can already navigate traffic autonomously. And there is much more to landing men on the moon than physics. Check out Tesla's live shipping code: http://jalopnik.com/teslas-autopilot-system-is-awesome-and-creepy-and-a-sig-1736573089 http://jalopnik.com/teslas-autopilot-system-is-awesome-and-c... Are you predicting that cars will never be fully autonomous?
  - agumonkey 11y ago
    As a website commenter I'm not sure I should predict anything. To explain a bit more my previous message, as complex as going to the moon was, it seems a problem scientists knew how to define. One vehicle, 2 planets, the system is describable. Traffic with lots of agents potentially interacting with every other agent, hence my chaos analogy, lots and lots of variable, ... is blurry. There's successful research to describe multi agents collision avoidance but I've never seen it used on the field yet. ps: reminds me airplanes vs car accident rates. Even if flying is inherently complex in itself, there's almost no traffic and almost no obstacle in air. Nobody would crash a car alone in the desert.

sklogic 11y ago

His 40 years of experience obviously did not include the only recently available GPUs and ASIC deep neural networks which are at the heart of the current self-driving robotics research. There is a game changer, and any previous experience is irrelevant.

spullara 11y ago

They are going to be interviewing him in a driverless car in 10 years and asking where he got it wrong.

pgodzin 11y ago

I think the difference between self-driving cars and Apollo program/subs examples is that the cars can be tested much more thoroughly. No matter how good your simulators are for space conditions, you're never going to really know how it's going to work in actual conditions. Self-driving cars can and are logging millions of miles in a variety of conditions, and every common and almost every remote case will be dealt with before launched to the general public.

highfreq 11y ago

40 years of history showing failure, of course for most of that time they have had between a millionth and half the computing power we have today, and most of those years had now where near the capability of image sensors and other sensors. So how much weight should we really give history on something like this?

simonh 11y ago
I don't think they're arguing against automation, just that completely driverless vehicles won't be the optimum path. I think there will be some applications for which fully driverless vehicles will be fine, such as trucks transporting goods between loading bays and taxis collecting and dropping off at street addresses, and many others where some level of driver control will be useful. Take their example of submarines. Fully autonomous submarines are completely possible, but what they point out is that some level of human control is more useful. They don't say why, but my guess is that this is because if the sub comes across something unusual that merits further investigation, humans are better at making that judgement call than a robot would. The same principle applies to the Mars rovers. But it also applies to plenty of routine uses of vehicles. Most of the time I'm driving from my house to a car park or street parking space and such journeys are probably completely automatable. But sometimes I arrive at e.g. a country fair and the parking is in a field and I know I want to park over there in a place which will soon be shaded by that tree, and not in a different place where it looks a bit muddy. At times like that I would want much more direct control over the vehicle and an automated system would not be able to make such decisions for me. So while the article does a poor job saying so, I think the point is that 100% full driverless automation isn't the best answer in every situation. It might be a fine solution for many purposes, but not all. They're arguing against a (perhaps imagined and possibly straw man) maximalist automation position such as 'in 20 years time no cars will have manual controls'.

meesterdude 11y ago

Here's the thing: we can do anything we want. And there is a huge demand for fully driverless cars that nobody is going to be able to stop, despite whatever sound and reasoned advice they may have. It's coming, like it or not.

rebootthesystem 11y ago

> “That’s just proven to be a loser of an approach in a lot of other domains,” Mindell says. “I’m not arguing this from first principles. There are 40 years’ worth of examples.” Well, that's the problem. Everything the author says is supported by the last N decades in engineering. That argument is so flawed it is astounding that it is coming out of someone from MIT. In looking at his bio it seems he has dedicated quite a bit of time to teaching and researching the history of engineering. It might just be the case that he isn't fully up to date on topics relevant to automation. Does he have an up-to-date CS/AI background? How about computing or chip design? Saying that automation has failed to deliver in the past N decades is like the Wright Brothers saying "let's not bother because powered flight has failed to be developed in the past N decades". That is not how engineering works. Today hundreds of millions of people have phones in their pockets that are orders of magnitude more powerful than the computers decades ago. We have better, smaller, cheaper sensors of all kinds. We have accumulated knowledge in relevant CS and AI fields. Today we have cars that can actually drive themselves to various degrees under their own computing power (without having to communicate with a supercomputer in a building). And this is just the beginning. Fully auto-drive cars will happen. Of course, they will retain the ability to be human driven as well for some time, probably decades. There are many reason for which that makes sense. I am fairly confident we will get to the point where the average self-driver will be safer than the average human driver. All you have to do is point at a teenager behind the wheel. Commutes will be shorter (due to driving efficiently using data) and safer and, with electric cars, healthier (you won't have to breathe the crap coming out of all those exhaust pipes). Besides, think of the wasted potential of millions of people staring at a road for hours rather than doing something more useful. If you commute one hour per day you'll get back 250 hours per year.