10 ms·

Former Uber Backup Driver: 'We Saw This Coming'

121 points by mgiannopoulos 9y ago

twblalock 9y ago
This isn't just about Uber -- all semi-autonomous cars are going to have this problem. The people behind the wheel will become distracted, increasingly so as the technology becomes better and they need to intervene less and less often. Even people who are trained to pay attention while testing self-driving cars become distracted, so we can assume the general public will become distracted too. The only viable autonomous cars are ones that will never require human intervention, because humans aren't able to intervene effectively and may make situations worse as they snap out of a distracted state. But we can't get to full autonomy without first testing the technology, and no artificial test conditions could replicate the complexity of the real world. Society will either learn to accept some avoidable deaths during the testing phase or will ban the technology. I suspect a few more incidents will result in a ban, and we will not have fully autonomous cars for a very long time.
- mc32 9y ago
  While I agree with much of what you say, I think we can have autonomous vehicles without having to be 100% independent of human intervention. It will require an effective "early warning system" which foresees issues it may not be able to process --however, this will require more inter-automobile communication and extravehicular communication (GPS, other inputs). Relying on only what the vehicle has on-board will probably not be good enough, for now. These supporting systems would communicate "trouble ahead" (ambiguous information, incomplete information, lack of information, unknown information, etc.)
  - snaily 9y ago
    I think you're describing SAE level 4, while the parent is arguing for level 3 being unsafe. Level 3 means the user has to take over from an active driving situation within a pre-specified time period, whereas level 4 means the car will always be able to hand over in a "safe" state (an thus with unbounded response time).
- agumonkey 9y ago
  As other people sometime say, there can be self driving slow vehicles. For elders, for tourism, for near-home moving of things or just parking/unparking. In the meantime level 5 auto automation can mature in test areas.
  - akira2501 9y ago
    > there can be self driving slow vehicles. This is not a safe option, particularly on certain roadways. Accidents increase when you have a higher levels of relative speed difference between lanes of traffic. Having "turtle mode AI cars" isn't going to help us. Speaking of the elderly.. they often end up in fatal accidents even at low speeds. Their frail bodies combined with strong safety restraints are not a good combination; particularly in the rear seats where their level of risk increases to nearly 2x the front seats.
    - agumonkey 9y ago
      I meant reaaally slow. Think not-road-accepted slow. Golf cart speeds at 'worst'. Maybe these aren't cars. But you can still transport some stuff and people in specific places and usage. Otherwise I agree with you.
      - nradov 9y ago
        You're referring to Neighborhood Electric Vehicles. Those operate under special rules in many areas. https://en.wikipedia.org/wiki/Neighborhood_Electric_Vehicle https://en.wikipedia.org/wiki/Neighborhood_Electric_Vehicle
  - ramphastidae 9y ago
    Great comment. Uber's behavior shows it is interested in securing a monopoly of local transportation markets, not self-driving technology. Self-driving technology is a means to that end. If they cared about self-driving technology itself they would be starting with much smaller applications and working their way up rather than forcing themselves on to our streets as quickly as possible before their funding runs out.
- snaily 9y ago
  Also known as the "Level 3 is inherently unsafe" argument. Waymo noticed this back in 2012: https://driverless.wonderhowto.com/news/waymo-was-right-why-every-car-maker-should-skip-level-3-0178497/ https://driverless.wonderhowto.com/news/waymo-was-right-why-...
  - digi_owl 9y ago
    In case others are wondering about the details about the levels: https://en.wikipedia.org/wiki/Autonomous_car#Classification https://en.wikipedia.org/wiki/Autonomous_car#Classification
  - stale2002 9y ago
    Well, so level 3 can be safe under certain conditions and still be world changing. For example, imagine a self driving truck that works perfectly, but only during the day, on highways, when it is not raining. Such a system would still put millions out of work, as firing half of the truck drivers is possible if something works when it is sunny, on the highway.
    - rco8786 9y ago
      I’ve been pretty convinced for a while that this is effectively how it will play out. Effectively the “iRobot” model of designated areas/highways that are autonomous and highly controlled/protected and surface streets will remain human controlled for a long time.
    - tjoff 9y ago
      At least level 4 is required for that.
    - twblalock 9y ago
      Truck drivers do a lot more than just drive trucks: http://marginalrevolution.com/marginalrevolution/2018/02/will-truckers-automated-comments.html http://marginalrevolution.com/marginalrevolution/2018/02/wil...
      - 9y ago
- digi_owl 9y ago
  I wonder if we can gain any insights from experiences with airplane autopilots.
  - greenyoda 9y ago
    Airplane autopilots operate in a completely different environment, in which planes' flight paths are set and monitored by Air Traffic Control. A plane on autopilot would never have to dodge a pedestrian, and would be separated from other planes by thousands of feet.
    - akira2501 9y ago
      > in which planes' flight paths are set They're 'set' in the sense that if communications is lost between the pilot and ATC, there's a pre-determined and more importantly pre-authorized path for the aircraft to fly through. > A plane on autopilot would never have to dodge a pedestrian It happens quite a bit, which is why we invented TCAS. [1] A common misconception is that _all_ traffic is required to file a flight plan, and that's only true for IFR (instrument) traffic and not for VFR (visual) traffic. So, an IFR flight can, and will, encouter VFR flights that interfere with their flight. > and would be separated from other planes by thousands of feet. In certain situations, in transit regions and during clearance/delivery, this is not true. 1: https://en.wikipedia.org/wiki/Traffic_collision_avoidance_system https://en.wikipedia.org/wiki/Traffic_collision_avoidance_sy...
      - shanghaiaway 9y ago
        An airplane has never had to dodge a pedestrian.
        jlgaddis 9y ago
        Since you said "never", I'll just mention that there are videos on Youtube from skydivers who would disagree with you.
        9y ago
- jay_kyburz 9y ago
  > But we can't get to full autonomy without first testing the technology, and no artificial test conditions could replicate the complexity of the real world. Actually, I think a test environment could be a lot more complex. It's mind blowing to me that there doesn't seem to be some big track somewhere with automated crash test dummies riding their bikes across the street in the dark. Automated cars running red lights and stop signs. Automated accidents in front of self driving cars. Automated kids chasing a football across the road. It's mind blowing to me that governments do not require self driving cars do well on all these test before they allow the cars on the road. Governments should prepare these test, and car technology should complete the tests before they are allowed on the road. And when a car fails any real world situation, we can step back and create new tests. Really: We can test to any complexity we want. A plan lading on the road? A flood? A heard of sheep or flock of birds? A dust storm or other weather than might prevent the cars sensors working.
  - TAForObvReasons 9y ago
    Proper development requires a conservative and methodical approach. Uber is racing to be the first, rushing to launch before properly testing cases like those you enumerated. As they would say, "Move Fast and Break Things"
    - twodave 9y ago
      People
  - as2587 9y ago
    fyi -- people are working on this: https://mcity.umich.edu/ https://mcity.umich.edu/
  - inverse_pi 9y ago
    There are test tracks. Both Waymo and Uber have publicly said that they have fake cities to test the code https://www.theatlantic.com/technology/archive/2017/08/inside-waymos-secret-testing-and-simulation-facilities/537648/ https://www.theatlantic.com/technology/archive/2017/08/insid... http://www.businessinsider.com/ubers-fake-city-pittsburgh-self-driving-cars-2017-10 http://www.businessinsider.com/ubers-fake-city-pittsburgh-se...
- nradov 9y ago
  Most autonomous vehicle companies are approaching the problem backwards. They're trying to build a vehicle where the computer is driving until it gets confused, and then asks a human (either in the vehicle or remote operator) to take over with little notice in an emergency. But we could get greater safety benefits sooner by having the human driver always drive with the computer overriding control inputs when it detects an unsafe situation. The basic concept is already proven with stability control systems and, more recently, front collision avoidance (automatic braking) systems. We should focus on expanding and extending those systems to cover more situations and the full range of vehicle control axes.
  - newnewpdro 9y ago
    There's more money and power to be had by taking control away from people. Fundamentally that is what's going on here. This is a massive power grab. This is convincing people to give up control of their transportation under a facade of improved safety and convenience. Imagine streets full of powerful multi-ton wheeled drones relatively few people have control over the software operating. Ignoring conspiracy scenarios, just imagine how 0-day hacks of that system manifest. None of these situations are possible until you convince everyone to take ownership of one of these robots, or at least accept and utilize a ridesharing fleet of them.
  - 05 9y ago
    That's impossible in reality, because computer doesn't have God's view of the environment - it can only predict things with a degree of confidence. It can't say for sure whether you get T-boned at an intersection you enter at 35MPH the moment the light turns green, so it won't reliably save you from a collision. A fully self driving system, on the other hand, won't enter the intersection without waiting 1-2 seconds, and would lower its speed below the speed limit when visibility is limited. A system that constantly keeps braking 'for no reason' won't get very far in the real world, because people would be very frustrated with it.
    - nradov 9y ago
      It's not at all impossible. It's already being done. Existing vehicles with forward collision avoidance systems are working well today. We'll never reach zero risk and you can always contrive some scenarios where automated systems wouldn't help. But it's still worth pursuing incremental improvements. For example, street sign recognition is getting pretty good. In a few years it should be possible to build a system that detects when the driver is about to run through a stop light or sign, and automatically brake.
- deleted 9y ago
  [deleted]
- BurningFrog 9y ago
  Surely this is a solved problem in several other contexts/industries?
- joshuak 9y ago
  Human attention vs low probably events is a known and solved problem. I've heard the TSA does this. You simply introduce fake events into the system at a frequency judged to cause human operators to stay alert, and responsive.
- WalterSear 9y ago
  They don't >have< to have this problem. The problem is that the robot should be watching the person, not vice-versa. This however, can't be sold as 'almost self-driving' to a gullible public.
- icc97 9y ago
  I think however the other way round does work. That is computers do stay alert and are able to take over with very short notice. This is essentially human augmentation. The regular collision detection in many cars now is a good example. It's a relatively simple system that should stop a distracted driver from crashing. Musk said something similar for the Tesla trucks, the auto-pilot can take over if the driver falls asleep or becomes unwell. Again this is augmentation. The only way to keep the driver alert is to have them driving all the time.
mgiannopoulos 9y ago
What I don’t understand: if Uber was firing people for using their mobiles in the car, which did they allow them to have a mobile phone while on the job?
- gwbas1c 9y ago
  It's plainly stated in the article: for emergencies.
  - mgiannopoulos 9y ago
    Surely they can provide a mobile phone with just the Uber app and ability to call though.
- greenyoda 9y ago
  - News reports have mentioned that Uber was using some of their driverless cars to pick up passengers, for which the driver would need access to the Uber app. - You want the driver to be able to call 911 if they're involved in an accident, or are the victim of a crime, or have a medical emergency. Presumably, they'd only be firing people for using their phones while the car was driving itself, and they were supposed to be supervising it. It would be OK to use a phone while the car wasn't moving.
  - martinald 9y ago
    It's pretty ironic though that Uber's driver app almost/does require you to use it in motion though, eg when you are driving and another fare comes up. Nearly every uber drive I see goes over to the phone and slides to accept the fare.
  - mgiannopoulos 9y ago
    They don't need to access any other apps though. Uber could provide a locked device allowing only emergency calls and the Uber app.
maym86 9y ago
If incidents only occur when the driver is distracted then it will always look like the cause was driver distraction even if they were diligent for 99.99% of the trip. There will likely be a bias in the future for incidents like this to show some level of driver distraction. This makes it easy to blame the driver when the idea of a human as a supervisor is fundimentaly flawed.
_bxg1 9y ago
If it'd been a Waymo or Tesla car I'd have been more likely to give some benefit of the doubt. Driving is hard and most of these systems already make fewer mistakes than human drivers. But Uber never has been and never will be trustworthy. It's a toxic organization that will continue moving fast and breaking things - sometimes intentionally - until it's stopped by either the market or the government.
- akira2501 9y ago
  > Driving is hard and most of these systems already make fewer mistakes than human drivers. When driven in a small number of states that have beneficial weather conditions. I don't think we've done _nearly_ enough testing to come to this conclusion yet.
  - herbstein 9y ago
    Even so, the places that Tesla and Waymo primarily drive - while "easy" for an autonomous car - should then also be easier for human drivers. His point is that they're in fewer mistakes and human drivers. In the end that is all we need from these cars. As long as they make fewer mistakes than us they are a success.
    - jackvalentine 9y ago
      > Even so, the places that Tesla and Waymo primarily drive - while "easy" for an autonomous car - should then also be easier for human drivers. His point is that they're in fewer mistakes and human drivers. Genuinely curious if you've come across accident statistics that break down human miles in to similar categories as the kind of miles autonomous cars drive.
- cctt23 9y ago
  The numbers so far, including human intervention rates do not support the claim that autonomy beats humans. With a limited dataset, the suggestion is very much in the other direction. These vehicles may someday be better than us, but “someday” isn’t today. In particular I think Uber had a human intervention on avg every 13mi. Edit: I’d love to hear a counterpoint, although of course I accept that’s not required. Like the discussion about fusion, people seem to argue from a point in the future when all of the problems and limitations of today are gone. Let’s try arguing from what’s possible now instead.
JelteF 9y ago
These people are basically giving driving lessons, just to cars instead of people. Regular driving instructors can do their job fulltime in a safe way without getting distracted. They don't control the car most of the time, yet they still are able to stop it in case of a student error. So to me the question is: What is making this difference? Is it the level of training for the instructor? Human interaction? Is it the sign on their car that shows it's a lesson car to the other drivers? Or is it something else? PS, this post is based on the way driving lessons happen in the Netherlands. I'm not sure if it's completely comparable to the US.
- greenyoda 9y ago
  Driving instructors probably don't drive nearly as long without a break (each lesson is 30-60 minutes). They also probably have much more mental stimulation, since (1) they're talking to the student, (2) the student probably needs much more frequent intervention than a self-driving car does, and (3) they have a variety of different students each day, each of which has a different skill level and personality.
  - sitkack 9y ago
    The teacher also asks the student to do increasingly difficult challenges and actively provides feedback on their performance. The same should be done for self driving cars, millions of miles in ideal conditions is worthless. These cars need to be driven by bank robbers, gang members and stunt drivers for a Quentin Tarantino movie.
- exit 9y ago
  there is a very different kind of learning at work in autonomous cars vs humans. these people are "backups", not instructors giving explicit instruction to any student. and have you seen the vide of the "instructor" during the fatal uber crash? completely zoned out compared to what a human-instructing instructor would do.
- scott00 9y ago
  The two roles are not comparable in my opinion. In most cases an instructor of humans has no capability to control the car. I've heard of some cases where they have brakes installed on the passenger side, but I'm not sure how common that is. I've never heard of a teaching car with full duplicate controls, though it's certainly possible they exist. I think even a bad new human driver is still better than an Uber automated system at the moment.
rmason 9y ago
It would seem reasonable to me that once someone stepped off the curb that the human driver should get notified that there was someone in the street on their left. It would heighten their awareness and make it more likely they'd react if the car did not. Something as simple as right and left buzzers. Course if the lidar didn't detect the person at all then Uber's problems are much worse.
- augbog 9y ago
  Heck that sounds pretty useful even for current cars today. I imagine it isn't cheap though to add and even if it was reasonably priced I imagine they would make it optional in case people didn't want to add it to their car price in which case many would simply say "Nah I don't need that. I'm a great driver!"
  - kalleboo 9y ago
    These kinds of things have existed for a while now https://newatlas.com/volvo-s60-pedestrian-safety-system/13589/ https://newatlas.com/volvo-s60-pedestrian-safety-system/1358...
- theclaw 9y ago
  Trucks already have these kind of audible warnings when vehicles or cyclists are in their blind spot. The problem in Tempe was that the car did not recognise the pedestrian, so how could it be relied upon to warn the driver? Perhaps a better option would be to buzz when the driver is not looking at the road, as was the case in the Tempe accident. This technology is already in use [0]. [0] https://en.wikipedia.org/wiki/Driver_drowsiness_detection https://en.wikipedia.org/wiki/Driver_drowsiness_detection
- LorenPechtel 9y ago
  I think this is going to turn out to be the fundamental issue--that the computer wasn't programmed to address illegal actions. The car has the right of way, it didn't consider she might enter the space anyway.
collingreene 9y ago
The two people interviewed were fired for cause from this same program, of course they will have a negative opinion. One even fired for the same thing this safety driver failed to do. >Both Kelley and the former driver in Tempe were dismissed from their jobs with Uber earlier this year for safety infractions: Kelley said he was let go after rolling through a stop sign while he was operating the car, which he disputes; the individual in Tempe said he was dismissed for using his phone while the vehicle was in motion. The bit about level 3 considered harmful makes a lot of sense and isn't something that I would have intuitively thought of
- shanghaiaway 9y ago
  I guess you missed the point of TFA. It is impossible for humans to perform the tasks these two persons were hired to do. 100% of people who attempt to do that job will eventually fail, likely sooner rather than later.
  - Aspyre 9y ago
    Wrong. At least 95% of the people who made the cut and didn't get fired after a few weeks handled being the only operator of the vehicle with ease.
    - icebraining 9y ago
      And they will continue to handle it with ease right up to the moment when they hit another cyclist. What makes you so confident that the mere two years of running the program is enough to reliably calculate that number, when drivers like the one in the article managed to last more than a year until they got fired?
- jacquesm 9y ago
  If trained and well disciplined airline pilots and railway engineers have problems staying alert in situations like this you can bet that your average Uber 'backup driver' (what a job title) isn't going to be any better.
icc97 9y ago
"noncompliant actors" is the Uber term for pedestrians in the road. I understand the computer science concept to abstract the term but when you're talking to the drivers of the cars this is in retrospect an awful term. Rather depressingly all I can think of is the ED 2000 from Robocop that shoots citizens who don't comply.
- maym86 9y ago
  Where did you hear this? The Caltrain calls incidents where it hits a pedestrian "tresspasser strikes". Always thought this was awful.
  - icc97 9y ago
    From the article: > In it, they are trained to keep hands hovering near the wheel at all times so that they can quickly take control when the car does not safely respond to dangerous road conditions or “noncompliant actors,” such as people walking in the roadway
- deleted 9y ago
  [deleted]
cma 9y ago
Rather than audit dashcam footage after the fact for infractions related to attention and phone use, these companies need realtime remote monitoring of the drivers (and realtime AI driven monitoring for areas without signal and attention lapses by the remote monitor employees).
- shanghaiaway 9y ago
  Lmao, so now you need two people to drive a car.
  - cma 9y ago
    While developing immature technology, yes. Maybe three people total (driver, and someone monitoring the driver, and someone else monitoring the scene from a camera up on a small boom pole).
intended 9y ago
Autonomous cars are not coming. A certain amount of over familiarity with western driving conditions blinds these attempts to the reality that there are two worlds on the roads - the legal and socially manufactured layer and reality. Which is why looking at the actual problem space, like nations where road rules tend to be ignored, makes it clear that what you actually have to solve for. It has been highlighted before - but the autonomous car can only work if the road itself is active / aware
- sharpercoder 9y ago
  Societies (and therefore roadsystems) will adapt, because speed and safety. Roads will be changed to cater to the needs of self-driving cars; smart traffic lights, clear lining, clear signage, less leveled intersections, dedicated lanes, I can think of dozens more changes which make it easier and safer and more deterministic for computers to drive cars. Development of these changes will be spurred once self-driving cars can make more efficient use of the road; the immense pressure of society to get rid of traffic jams will make sure of that. Maybe some nations won't implement these things nationwide with 100% coverage, but they certainly will implement changes for certain roads. Highway roads first, then collector roads, and lastly neighborhood roads.
  - jlgaddis 9y ago
    In America, at least, many (most?) roads and highways are already neglected and getting worse. This is, so we hear, due to a lack of money to keep our roads properly maintained. If we don't have the money to do that, where is the money to retro-fit every road in the country to accomodate autonomous vehicles going to come from?
    - skellera 9y ago
      It's a good question since most of those cars are electric. Taxes typically taken in at the gas pump are being skirted around by having electric cars. Do we just start charging MUCH more for registration? Especially cause these cars have the potential to operate most of the day (except when charging) so they will be causing more wear than a normal car.
twodave 9y ago
In my opinion, automation is a way better use case for mass transit than 2-5 passenger consumer vehicles. These companies ought to be focusing on cities where population is growing but density is low (one example would be Jacksonville, FL). Start automating buses that run predictable routes and fire all those city employee bus drivers who are driving up property taxes because they need a pension.
g09980 9y ago
In Japanese rail, conductors and drivers periodically point with their hand [1][2] to indicate awareness of the task. Reading the article, perhaps these distracted test drivers could use a similar approach. [1] https://en.wikipedia.org/wiki/Pointing_and_calling https://en.wikipedia.org/wiki/Pointing_and_calling [2] https://www.youtube.com/watch?v=9LmdUz3rOQU https://www.youtube.com/watch?v=9LmdUz3rOQU
- PherricOxide 9y ago
  Huh, I had a bus driver in Seattle (not Japanese) that did something similar. He would point at and call out all the major intersections, speed limit signs, etc. I honestly thought it was some kind of neurosis/obsessive compulsive disorder, but seeing this exact same behavior makes me wonder if he actually picked it up as a trick to increase is awareness on driving.
  - Izkata 9y ago
    This is one of the strategies they use on some of the drivers in the TV show "Canada's Worst Driver", to get the ones scared of driving to regain control, convince the distracted ones to watch the road, and teach the ones who don't know where to look how to look at everything. (They don't do any pointing though, because that would require taking their hands off the wheel)
mentos 9y ago
Seems like they need another system to monitor the backup driver to make sure their eyes are on the road.
frankydp 9y ago
Shouldn't test vehicles have at least the same level of attention monitors as current auto pilot/assist vehicles do? For these specific vehicles shouldn't they have some hand placement sensors or wifi strength detectors or mobile signal detectors or eyeball monitoring at a minimum? Or use a custom app to "lock" the phone while driving, exit app and the car pulls over. Not that the driver would have prevented this incident, but seems like a logical requirement.

121 points by mgiannopoulos 9y ago

This isn't just about Uber -- all semi-autonomous cars are going to have this problem. The people behind the wheel will become distracted, increasingly so as the technology becomes better and they need to intervene less and less often. Even people who are trained to pay attention while testing self-driving cars become distracted, so we can assume the general public will become distracted too. The only viable autonomous cars are ones that will never require human intervention, because humans aren't able to intervene effectively and may make situations worse as they snap out of a distracted state. But we can't get to full autonomy without first testing the technology, and no artificial test conditions could replicate the complexity of the real world. Society will either learn to accept some avoidable deaths during the testing phase or will ban the technology. I suspect a few more incidents will result in a ban, and we will not have fully autonomous cars for a very long time.

mc32 9y ago
While I agree with much of what you say, I think we can have autonomous vehicles without having to be 100% independent of human intervention. It will require an effective "early warning system" which foresees issues it may not be able to process --however, this will require more inter-automobile communication and extravehicular communication (GPS, other inputs). Relying on only what the vehicle has on-board will probably not be good enough, for now. These supporting systems would communicate "trouble ahead" (ambiguous information, incomplete information, lack of information, unknown information, etc.)
- snaily 9y ago
  I think you're describing SAE level 4, while the parent is arguing for level 3 being unsafe. Level 3 means the user has to take over from an active driving situation within a pre-specified time period, whereas level 4 means the car will always be able to hand over in a "safe" state (an thus with unbounded response time).
agumonkey 9y ago
As other people sometime say, there can be self driving slow vehicles. For elders, for tourism, for near-home moving of things or just parking/unparking. In the meantime level 5 auto automation can mature in test areas.
- akira2501 9y ago
  > there can be self driving slow vehicles. This is not a safe option, particularly on certain roadways. Accidents increase when you have a higher levels of relative speed difference between lanes of traffic. Having "turtle mode AI cars" isn't going to help us. Speaking of the elderly.. they often end up in fatal accidents even at low speeds. Their frail bodies combined with strong safety restraints are not a good combination; particularly in the rear seats where their level of risk increases to nearly 2x the front seats.
  - agumonkey 9y ago
    I meant reaaally slow. Think not-road-accepted slow. Golf cart speeds at 'worst'. Maybe these aren't cars. But you can still transport some stuff and people in specific places and usage. Otherwise I agree with you.
    - nradov 9y ago
      You're referring to Neighborhood Electric Vehicles. Those operate under special rules in many areas. https://en.wikipedia.org/wiki/Neighborhood_Electric_Vehicle https://en.wikipedia.org/wiki/Neighborhood_Electric_Vehicle
- ramphastidae 9y ago
  Great comment. Uber's behavior shows it is interested in securing a monopoly of local transportation markets, not self-driving technology. Self-driving technology is a means to that end. If they cared about self-driving technology itself they would be starting with much smaller applications and working their way up rather than forcing themselves on to our streets as quickly as possible before their funding runs out.
snaily 9y ago
Also known as the "Level 3 is inherently unsafe" argument. Waymo noticed this back in 2012: https://driverless.wonderhowto.com/news/waymo-was-right-why-every-car-maker-should-skip-level-3-0178497/ https://driverless.wonderhowto.com/news/waymo-was-right-why-...
- digi_owl 9y ago
  In case others are wondering about the details about the levels: https://en.wikipedia.org/wiki/Autonomous_car#Classification https://en.wikipedia.org/wiki/Autonomous_car#Classification
- stale2002 9y ago
  Well, so level 3 can be safe under certain conditions and still be world changing. For example, imagine a self driving truck that works perfectly, but only during the day, on highways, when it is not raining. Such a system would still put millions out of work, as firing half of the truck drivers is possible if something works when it is sunny, on the highway.
  - rco8786 9y ago
    I’ve been pretty convinced for a while that this is effectively how it will play out. Effectively the “iRobot” model of designated areas/highways that are autonomous and highly controlled/protected and surface streets will remain human controlled for a long time.
  - tjoff 9y ago
    At least level 4 is required for that.
  - twblalock 9y ago
    Truck drivers do a lot more than just drive trucks: http://marginalrevolution.com/marginalrevolution/2018/02/will-truckers-automated-comments.html http://marginalrevolution.com/marginalrevolution/2018/02/wil...
    - 9y ago
digi_owl 9y ago
I wonder if we can gain any insights from experiences with airplane autopilots.
- greenyoda 9y ago
  Airplane autopilots operate in a completely different environment, in which planes' flight paths are set and monitored by Air Traffic Control. A plane on autopilot would never have to dodge a pedestrian, and would be separated from other planes by thousands of feet.
  - akira2501 9y ago
    > in which planes' flight paths are set They're 'set' in the sense that if communications is lost between the pilot and ATC, there's a pre-determined and more importantly pre-authorized path for the aircraft to fly through. > A plane on autopilot would never have to dodge a pedestrian It happens quite a bit, which is why we invented TCAS. [1] A common misconception is that _all_ traffic is required to file a flight plan, and that's only true for IFR (instrument) traffic and not for VFR (visual) traffic. So, an IFR flight can, and will, encouter VFR flights that interfere with their flight. > and would be separated from other planes by thousands of feet. In certain situations, in transit regions and during clearance/delivery, this is not true. 1: https://en.wikipedia.org/wiki/Traffic_collision_avoidance_system https://en.wikipedia.org/wiki/Traffic_collision_avoidance_sy...
    - shanghaiaway 9y ago
      An airplane has never had to dodge a pedestrian.
      - jlgaddis 9y ago
        Since you said "never", I'll just mention that there are videos on Youtube from skydivers who would disagree with you.
        9y ago
jay_kyburz 9y ago
> But we can't get to full autonomy without first testing the technology, and no artificial test conditions could replicate the complexity of the real world. Actually, I think a test environment could be a lot more complex. It's mind blowing to me that there doesn't seem to be some big track somewhere with automated crash test dummies riding their bikes across the street in the dark. Automated cars running red lights and stop signs. Automated accidents in front of self driving cars. Automated kids chasing a football across the road. It's mind blowing to me that governments do not require self driving cars do well on all these test before they allow the cars on the road. Governments should prepare these test, and car technology should complete the tests before they are allowed on the road. And when a car fails any real world situation, we can step back and create new tests. Really: We can test to any complexity we want. A plan lading on the road? A flood? A heard of sheep or flock of birds? A dust storm or other weather than might prevent the cars sensors working.
- TAForObvReasons 9y ago
  Proper development requires a conservative and methodical approach. Uber is racing to be the first, rushing to launch before properly testing cases like those you enumerated. As they would say, "Move Fast and Break Things"
  - twodave 9y ago
    People
- as2587 9y ago
  fyi -- people are working on this: https://mcity.umich.edu/ https://mcity.umich.edu/
- inverse_pi 9y ago
  There are test tracks. Both Waymo and Uber have publicly said that they have fake cities to test the code https://www.theatlantic.com/technology/archive/2017/08/inside-waymos-secret-testing-and-simulation-facilities/537648/ https://www.theatlantic.com/technology/archive/2017/08/insid... http://www.businessinsider.com/ubers-fake-city-pittsburgh-self-driving-cars-2017-10 http://www.businessinsider.com/ubers-fake-city-pittsburgh-se...
nradov 9y ago
Most autonomous vehicle companies are approaching the problem backwards. They're trying to build a vehicle where the computer is driving until it gets confused, and then asks a human (either in the vehicle or remote operator) to take over with little notice in an emergency. But we could get greater safety benefits sooner by having the human driver always drive with the computer overriding control inputs when it detects an unsafe situation. The basic concept is already proven with stability control systems and, more recently, front collision avoidance (automatic braking) systems. We should focus on expanding and extending those systems to cover more situations and the full range of vehicle control axes.
- newnewpdro 9y ago
  There's more money and power to be had by taking control away from people. Fundamentally that is what's going on here. This is a massive power grab. This is convincing people to give up control of their transportation under a facade of improved safety and convenience. Imagine streets full of powerful multi-ton wheeled drones relatively few people have control over the software operating. Ignoring conspiracy scenarios, just imagine how 0-day hacks of that system manifest. None of these situations are possible until you convince everyone to take ownership of one of these robots, or at least accept and utilize a ridesharing fleet of them.
- 05 9y ago
  That's impossible in reality, because computer doesn't have God's view of the environment - it can only predict things with a degree of confidence. It can't say for sure whether you get T-boned at an intersection you enter at 35MPH the moment the light turns green, so it won't reliably save you from a collision. A fully self driving system, on the other hand, won't enter the intersection without waiting 1-2 seconds, and would lower its speed below the speed limit when visibility is limited. A system that constantly keeps braking 'for no reason' won't get very far in the real world, because people would be very frustrated with it.
  - nradov 9y ago
    It's not at all impossible. It's already being done. Existing vehicles with forward collision avoidance systems are working well today. We'll never reach zero risk and you can always contrive some scenarios where automated systems wouldn't help. But it's still worth pursuing incremental improvements. For example, street sign recognition is getting pretty good. In a few years it should be possible to build a system that detects when the driver is about to run through a stop light or sign, and automatically brake.
deleted 9y ago
[deleted]
BurningFrog 9y ago
Surely this is a solved problem in several other contexts/industries?
joshuak 9y ago
Human attention vs low probably events is a known and solved problem. I've heard the TSA does this. You simply introduce fake events into the system at a frequency judged to cause human operators to stay alert, and responsive.
WalterSear 9y ago
They don't >have< to have this problem. The problem is that the robot should be watching the person, not vice-versa. This however, can't be sold as 'almost self-driving' to a gullible public.
icc97 9y ago
I think however the other way round does work. That is computers do stay alert and are able to take over with very short notice. This is essentially human augmentation. The regular collision detection in many cars now is a good example. It's a relatively simple system that should stop a distracted driver from crashing. Musk said something similar for the Tesla trucks, the auto-pilot can take over if the driver falls asleep or becomes unwell. Again this is augmentation. The only way to keep the driver alert is to have them driving all the time.

mgiannopoulos 9y ago

What I don’t understand: if Uber was firing people for using their mobiles in the car, which did they allow them to have a mobile phone while on the job?

gwbas1c 9y ago
It's plainly stated in the article: for emergencies.
- mgiannopoulos 9y ago
  Surely they can provide a mobile phone with just the Uber app and ability to call though.
greenyoda 9y ago
- News reports have mentioned that Uber was using some of their driverless cars to pick up passengers, for which the driver would need access to the Uber app. - You want the driver to be able to call 911 if they're involved in an accident, or are the victim of a crime, or have a medical emergency. Presumably, they'd only be firing people for using their phones while the car was driving itself, and they were supposed to be supervising it. It would be OK to use a phone while the car wasn't moving.
- martinald 9y ago
  It's pretty ironic though that Uber's driver app almost/does require you to use it in motion though, eg when you are driving and another fare comes up. Nearly every uber drive I see goes over to the phone and slides to accept the fare.
- mgiannopoulos 9y ago
  They don't need to access any other apps though. Uber could provide a locked device allowing only emergency calls and the Uber app.

maym86 9y ago

If incidents only occur when the driver is distracted then it will always look like the cause was driver distraction even if they were diligent for 99.99% of the trip. There will likely be a bias in the future for incidents like this to show some level of driver distraction. This makes it easy to blame the driver when the idea of a human as a supervisor is fundimentaly flawed.

_bxg1 9y ago

If it'd been a Waymo or Tesla car I'd have been more likely to give some benefit of the doubt. Driving is hard and most of these systems already make fewer mistakes than human drivers. But Uber never has been and never will be trustworthy. It's a toxic organization that will continue moving fast and breaking things - sometimes intentionally - until it's stopped by either the market or the government.

akira2501 9y ago
> Driving is hard and most of these systems already make fewer mistakes than human drivers. When driven in a small number of states that have beneficial weather conditions. I don't think we've done _nearly_ enough testing to come to this conclusion yet.
- herbstein 9y ago
  Even so, the places that Tesla and Waymo primarily drive - while "easy" for an autonomous car - should then also be easier for human drivers. His point is that they're in fewer mistakes and human drivers. In the end that is all we need from these cars. As long as they make fewer mistakes than us they are a success.
  - jackvalentine 9y ago
    > Even so, the places that Tesla and Waymo primarily drive - while "easy" for an autonomous car - should then also be easier for human drivers. His point is that they're in fewer mistakes and human drivers. Genuinely curious if you've come across accident statistics that break down human miles in to similar categories as the kind of miles autonomous cars drive.
cctt23 9y ago
The numbers so far, including human intervention rates do not support the claim that autonomy beats humans. With a limited dataset, the suggestion is very much in the other direction. These vehicles may someday be better than us, but “someday” isn’t today. In particular I think Uber had a human intervention on avg every 13mi. Edit: I’d love to hear a counterpoint, although of course I accept that’s not required. Like the discussion about fusion, people seem to argue from a point in the future when all of the problems and limitations of today are gone. Let’s try arguing from what’s possible now instead.

JelteF 9y ago

These people are basically giving driving lessons, just to cars instead of people. Regular driving instructors can do their job fulltime in a safe way without getting distracted. They don't control the car most of the time, yet they still are able to stop it in case of a student error. So to me the question is: What is making this difference? Is it the level of training for the instructor? Human interaction? Is it the sign on their car that shows it's a lesson car to the other drivers? Or is it something else? PS, this post is based on the way driving lessons happen in the Netherlands. I'm not sure if it's completely comparable to the US.

greenyoda 9y ago
Driving instructors probably don't drive nearly as long without a break (each lesson is 30-60 minutes). They also probably have much more mental stimulation, since (1) they're talking to the student, (2) the student probably needs much more frequent intervention than a self-driving car does, and (3) they have a variety of different students each day, each of which has a different skill level and personality.
- sitkack 9y ago
  The teacher also asks the student to do increasingly difficult challenges and actively provides feedback on their performance. The same should be done for self driving cars, millions of miles in ideal conditions is worthless. These cars need to be driven by bank robbers, gang members and stunt drivers for a Quentin Tarantino movie.
exit 9y ago
there is a very different kind of learning at work in autonomous cars vs humans. these people are "backups", not instructors giving explicit instruction to any student. and have you seen the vide of the "instructor" during the fatal uber crash? completely zoned out compared to what a human-instructing instructor would do.
scott00 9y ago
The two roles are not comparable in my opinion. In most cases an instructor of humans has no capability to control the car. I've heard of some cases where they have brakes installed on the passenger side, but I'm not sure how common that is. I've never heard of a teaching car with full duplicate controls, though it's certainly possible they exist. I think even a bad new human driver is still better than an Uber automated system at the moment.

rmason 9y ago

It would seem reasonable to me that once someone stepped off the curb that the human driver should get notified that there was someone in the street on their left. It would heighten their awareness and make it more likely they'd react if the car did not. Something as simple as right and left buzzers. Course if the lidar didn't detect the person at all then Uber's problems are much worse.

augbog 9y ago
Heck that sounds pretty useful even for current cars today. I imagine it isn't cheap though to add and even if it was reasonably priced I imagine they would make it optional in case people didn't want to add it to their car price in which case many would simply say "Nah I don't need that. I'm a great driver!"
- kalleboo 9y ago
  These kinds of things have existed for a while now https://newatlas.com/volvo-s60-pedestrian-safety-system/13589/ https://newatlas.com/volvo-s60-pedestrian-safety-system/1358...
theclaw 9y ago
Trucks already have these kind of audible warnings when vehicles or cyclists are in their blind spot. The problem in Tempe was that the car did not recognise the pedestrian, so how could it be relied upon to warn the driver? Perhaps a better option would be to buzz when the driver is not looking at the road, as was the case in the Tempe accident. This technology is already in use [0]. [0] https://en.wikipedia.org/wiki/Driver_drowsiness_detection https://en.wikipedia.org/wiki/Driver_drowsiness_detection
LorenPechtel 9y ago
I think this is going to turn out to be the fundamental issue--that the computer wasn't programmed to address illegal actions. The car has the right of way, it didn't consider she might enter the space anyway.

collingreene 9y ago

The two people interviewed were fired for cause from this same program, of course they will have a negative opinion. One even fired for the same thing this safety driver failed to do. >Both Kelley and the former driver in Tempe were dismissed from their jobs with Uber earlier this year for safety infractions: Kelley said he was let go after rolling through a stop sign while he was operating the car, which he disputes; the individual in Tempe said he was dismissed for using his phone while the vehicle was in motion. The bit about level 3 considered harmful makes a lot of sense and isn't something that I would have intuitively thought of

shanghaiaway 9y ago
I guess you missed the point of TFA. It is impossible for humans to perform the tasks these two persons were hired to do. 100% of people who attempt to do that job will eventually fail, likely sooner rather than later.
- Aspyre 9y ago
  Wrong. At least 95% of the people who made the cut and didn't get fired after a few weeks handled being the only operator of the vehicle with ease.
  - icebraining 9y ago
    And they will continue to handle it with ease right up to the moment when they hit another cyclist. What makes you so confident that the mere two years of running the program is enough to reliably calculate that number, when drivers like the one in the article managed to last more than a year until they got fired?
jacquesm 9y ago
If trained and well disciplined airline pilots and railway engineers have problems staying alert in situations like this you can bet that your average Uber 'backup driver' (what a job title) isn't going to be any better.

icc97 9y ago

"noncompliant actors" is the Uber term for pedestrians in the road. I understand the computer science concept to abstract the term but when you're talking to the drivers of the cars this is in retrospect an awful term. Rather depressingly all I can think of is the ED 2000 from Robocop that shoots citizens who don't comply.

maym86 9y ago
Where did you hear this? The Caltrain calls incidents where it hits a pedestrian "tresspasser strikes". Always thought this was awful.
- icc97 9y ago
  From the article: > In it, they are trained to keep hands hovering near the wheel at all times so that they can quickly take control when the car does not safely respond to dangerous road conditions or “noncompliant actors,” such as people walking in the roadway
deleted 9y ago
[deleted]

cma 9y ago

Rather than audit dashcam footage after the fact for infractions related to attention and phone use, these companies need realtime remote monitoring of the drivers (and realtime AI driven monitoring for areas without signal and attention lapses by the remote monitor employees).

shanghaiaway 9y ago
Lmao, so now you need two people to drive a car.
- cma 9y ago
  While developing immature technology, yes. Maybe three people total (driver, and someone monitoring the driver, and someone else monitoring the scene from a camera up on a small boom pole).

intended 9y ago

Autonomous cars are not coming. A certain amount of over familiarity with western driving conditions blinds these attempts to the reality that there are two worlds on the roads - the legal and socially manufactured layer and reality. Which is why looking at the actual problem space, like nations where road rules tend to be ignored, makes it clear that what you actually have to solve for. It has been highlighted before - but the autonomous car can only work if the road itself is active / aware

sharpercoder 9y ago
Societies (and therefore roadsystems) will adapt, because speed and safety. Roads will be changed to cater to the needs of self-driving cars; smart traffic lights, clear lining, clear signage, less leveled intersections, dedicated lanes, I can think of dozens more changes which make it easier and safer and more deterministic for computers to drive cars. Development of these changes will be spurred once self-driving cars can make more efficient use of the road; the immense pressure of society to get rid of traffic jams will make sure of that. Maybe some nations won't implement these things nationwide with 100% coverage, but they certainly will implement changes for certain roads. Highway roads first, then collector roads, and lastly neighborhood roads.
- jlgaddis 9y ago
  In America, at least, many (most?) roads and highways are already neglected and getting worse. This is, so we hear, due to a lack of money to keep our roads properly maintained. If we don't have the money to do that, where is the money to retro-fit every road in the country to accomodate autonomous vehicles going to come from?
  - skellera 9y ago
    It's a good question since most of those cars are electric. Taxes typically taken in at the gas pump are being skirted around by having electric cars. Do we just start charging MUCH more for registration? Especially cause these cars have the potential to operate most of the day (except when charging) so they will be causing more wear than a normal car.

twodave 9y ago

In my opinion, automation is a way better use case for mass transit than 2-5 passenger consumer vehicles. These companies ought to be focusing on cities where population is growing but density is low (one example would be Jacksonville, FL). Start automating buses that run predictable routes and fire all those city employee bus drivers who are driving up property taxes because they need a pension.

g09980 9y ago

In Japanese rail, conductors and drivers periodically point with their hand [1][2] to indicate awareness of the task. Reading the article, perhaps these distracted test drivers could use a similar approach. [1] https://en.wikipedia.org/wiki/Pointing_and_calling https://en.wikipedia.org/wiki/Pointing_and_calling [2] https://www.youtube.com/watch?v=9LmdUz3rOQU https://www.youtube.com/watch?v=9LmdUz3rOQU

PherricOxide 9y ago
Huh, I had a bus driver in Seattle (not Japanese) that did something similar. He would point at and call out all the major intersections, speed limit signs, etc. I honestly thought it was some kind of neurosis/obsessive compulsive disorder, but seeing this exact same behavior makes me wonder if he actually picked it up as a trick to increase is awareness on driving.
- Izkata 9y ago
  This is one of the strategies they use on some of the drivers in the TV show "Canada's Worst Driver", to get the ones scared of driving to regain control, convince the distracted ones to watch the road, and teach the ones who don't know where to look how to look at everything. (They don't do any pointing though, because that would require taking their hands off the wheel)

mentos 9y ago

Seems like they need another system to monitor the backup driver to make sure their eyes are on the road.

frankydp 9y ago

Shouldn't test vehicles have at least the same level of attention monitors as current auto pilot/assist vehicles do? For these specific vehicles shouldn't they have some hand placement sensors or wifi strength detectors or mobile signal detectors or eyeball monitoring at a minimum? Or use a custom app to "lock" the phone while driving, exit app and the car pulls over. Not that the driver would have prevented this incident, but seems like a logical requirement.