27 ms·
Arizona Government Suspends Uber's Self Driving Cars from Roads
- notatoad 9y agoSeems like kind of a pointless action to make it look like they're responding. Uber already suspended their testing in Arizona.
- suresk 9y agoFor all of the theoretical moral questions that have been brought up around self driving cars (the proverbial lever that lets you kill one person instead of a handful, for example), it's interesting/inevitable that this one came up first. I'm sure lots of people will argue that self-driving cars will, in the long run, save a lot of lives, and therefore we can justify some loss of life in the process. They are probably right about self-driving cars saving more lives in the long run, but I don't know that the conclusion follows. From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing and would not have been at-fault in the collision. But self-driving cars have more sensors and, in theory, more capabilities to prevent something like this, and we absolutely should expect them to do better. As someone with a little bit of experience with the machine learning side of things, I understand how this stuff is really hard, and I can imagine a handful of ways the system could have been confused, which makes it all the more important to understand what went wrong and how we could prevent it from happening in the future. As self-driving cars become more prevalent, things like this will happen and it will be tough to walk the line between over-reacting to every incident and killing progress in this area and designing safe cars that don't make the problem worse. I think it is prudent to have Uber stop testing on public roads until they can explain how this happened and how they can keep it from happening again.
- ars 9y ago> From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing No, a human would have easily avoided this (unless they were not looking). The video is much darker than what a person would see. Even when you can see the bike on that dark video, the car still does not react!
- MBCook 9y agoBased on what came out of the last few days I don’t think it’s clear at all, I think a human would not have hit the woman. The footage we saw was from a terrible dashcam and did not represent how dark the street actually was. Other people who have driven down the street with cameras show it to be perfectly reasonably lit, with streetlights and everything. Second, the woman was crossing across two lanes, so any driver watching would have already seen her cross the left-hand lane. It’s not like she popped out from behind a tree suddenly. She wasn’t even going that fast. This seems to be a massive failure of Uber’s system as well as a complete failure of the ‘safety’ driver. I don’t think anyone’s overreacting. I think they’re under reacting. I think it’s pretty clear over was heavily negligent. There’s no way they don’t get sued. What comes out at trial is going to be up amazing, I think. I think if this had been a “real” accident that would’ve been almost impossible to avoid, around it WAS impossible to avoid, or one where the human stepped in and it wasn’t good enough… We would be having a discussion we’re like what you’re saying. Maybe if this was what emai I think if this had been a “real“ accident that would’ve been almost impossible to avoid, around it WAS impossible to avoid, or one where the human stepped in and it wasn’t good enough… We would be having a discussion we’re like what you’re saying. Maybe if this was Waymo or one of the other comapnies. But it looks like the dubious honor of first death by a autonomous car was totally unnecessary and preventable.
- sokoloff 9y agoThe pedestrian also has a significant contribution of negligence to this collision, IMO.
- mjmahone17 9y agoWhy? They crossed a 35 mph (so in Tempe this is a “slow” road) at a corner (unmarked corners are also crosswalks, legally) on a well lit road. Where the car was many seconds away when she started crossing. Would a human driver be deemed at fault in a court of law? Usually not (though in this case, maybe) unless they kept going after the collision, or were under the influence. But that’s because we give human drivers a lot of leeway to kill people. Not because it isn’t their fault.
- dontreact 9y ago"From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing and would not have been at-fault in the collision." The camera footage is A LOT darker than what people have filmed there just a day or two after and is probably not a good representation of what it looked like to the human eye. Even if it was fairly accurate, the car would be HEAVILY and EXTREMELY dangerously overdriving its headlights in a way that humans would not do.
- takeda 9y ago> From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing and would not have been at-fault in the collision. But self-driving cars have more sensors and, in theory, more capabilities to prevent something like this, and we absolutely should expect them to do better. The footage was artificially darkened, or it was extremely crappy dash cam or both. I suspect Uber just tampered with it (to get some sympathy) Someone posted a footage from that place few days later and that is very well lit road. On Uber footage you can see lamps on the sky, yet it is very dark. Also note this dashcam is not what the car uses, and the car is also equipped with LIDAR so it should be able to spot pedestrian even in pitch black. > As self-driving cars become more prevalent, things like this will happen and it will be tough to walk the line between over-reacting to every incident and killing progress in this area and designing safe cars that don't make the problem worse. I think it is prudent to have Uber stop testing on public roads until they can explain how this happened and how they can keep it from happening again. Your are missing the point here. Preventing accidents like this should be thing that AI would excel at, even today. The fact that this failed means that there is something seriously wrong with how Uber is doing it.
- Aspyre 9y agoHow would the footage be tampered when it was taken by the police directly from the dashcam itself at the scene? Also, there was no moon on that night and all of the video comparisons that have been put out are low-light cameras. They are not very realistic. Also, noone seems to think it's a possibility that the agencies responsible for the lighting in this area to have made improvements immediately after the inicident before the public speculation started.
- ceejayoz 9y ago> all of the video comparisons that have been put out are low-light cameras If a self-driving car can't see 50 feet at night, maybe they should buy some low-light cameras? > Also, noone seems to think it's a possibility that the agencies responsible for the lighting in this area to have made improvements immediately after the inicident before the public speculation started. None of this matters. (I also find it pretty unlikely - the lights shown in Uber's video look plenty close together already to safely light the road. Even on an unlit road, the car's own headlights should've revealed the pedestrian far sooner.) A self-driving car should be able to see (via the car's headlights, infrared, LIDAR, etc.) a safe distance ahead of the car at all times. The car should detect the unsafe condition and refuse to continue - not mow down pedestrians. If it can't cope with a moonless, unlit road, it should not be on that road.
- zer00eyz 9y agoI'm going to take your post apart because I enjoyed it very much... > For all of the theoretical moral questions that have been brought up around self driving cars (the proverbial lever that lets you kill one person instead of a handful, for example), it's interesting/inevitable that this one came up first. Your talking about the "trolley problem" and it is a ethical question as opposed to moral. This is literally a fuckup, bad programing or poor design. We are going to have real ethical concerns because at some point a car is going to have to make a choice pedestrian or occupant - and I know that my elderly father would choose himself if he were in the car, but I might make a different choice.... will the cars ethics ever reflect my own? > I'm sure lots of people will argue that self-driving cars will, in the long run, save a lot of lives, and therefore we can justify some loss of life in the process. They are probably right about self-driving cars saving more lives in the long run, but I don't know that the conclusion follows. It is a tough one, and if there is more loss of life does using a self driving car a bit of fruit off a poison tree. https://en.wikipedia.org/wiki/Eduard_Pernkopf https://en.wikipedia.org/wiki/Eduard_Pernkopf springs to mind when you bring this up. > From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing Except the fact that when you look at footage of the road when driven on with a decent camera recording rather than the crapy low quality and very much artificially dark uber footage this looks to have been very avoidable. If the guy who was supposed to be there to take over had been looking up right then, we might not be talking about this at all... > As someone with a little bit of experience with the machine learning side of things, I understand how this stuff is really hard, and I can imagine a handful of ways the system could have been confused, which makes it all the more important to understand what went wrong and how we could prevent it from happening in the future. Were talking about uber here... it is also just as likely that they are in way over their heads. > As self-driving cars become more prevalent, things like this will happen and it will be tough to walk the line between over-reacting to every incident and killing progress in this area and designing safe cars that don't make the problem worse. When I read this bit I felt compelled to respond - the life of one victim is too many. In the early days of aviation people knew the risks... But this wasn't the test pilot who died this was a woman walking down the road. I look forward to the NTSB report - they tend to be the adults in these situations.
- 9y ago
- shakna 9y ago> From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing and would not have been at-fault in the collision. But self-driving cars have more sensors and, in theory, more capabilities to prevent something like this, and we absolutely should expect them to do better. Intel and Volvo tested against the terrible dashcam footage. If Uber hadn't disabled the XC90's safety systems, this wouldn't have happened. [0] The systems today can do better. [0] https://www.bloomberg.com/news/articles/2018-03-26/uber-disabled-volvo-suv-s-standard-safety-system-before-fatality https://www.bloomberg.com/news/articles/2018-03-26/uber-disa...
- asdfologist 9y agoLet’s suspend human driving too, since it kills over 100 people per day.
- vkou 9y agoHumans kill 1 person every ~100m miles. Uber has killed 1 person after ~3m miles. Yes, I would absolutely suspend a system that kills 3,000 people/day in favor of a system that kills 100 people/day.
- stagbeetle 9y agoThis is lying with statistics. To beat a dead statistician, everyone that's ever come in contact with air has died or will die.
- adjkant 9y agoIf you're going to claim it's a lie, citing evidence and why it's flawed would probably be a good start...
- stagbeetle 9y agoOut of the 95 billion or so[0] people that have ever breathed air before the year 1900, all are dead dead.[1] Every year on average, 500 people die of carbon monoxide poisoning[2]. If that rate was steady for the amount of time humans have been alive, that's a total of 25,950,000 deaths ever from carbon monoxide poisoning. "Carbon monoxide exposure kills 1 person every 18 hours (500/8766h). Oxygen has killed 1944 people every 18 hours (108x18)[3]. Yes, I would absolutely replace a gas that kills 155,000 (108x60x24) people/day[3] in favor of a gas that kills 1 person/day." This is lying with statistics. It's not the right way to use data. [0]https://www.prb.org/howmanypeoplehaveeverlivedonearth/ https://www.prb.org/howmanypeoplehaveeverlivedonearth/ [1]http://supercentenarian-research-foundation.org/TableE.aspx http://supercentenarian-research-foundation.org/TableE.aspx [2]https://www.cdc.gov/mmwr/preview/mmwrhtml/mm6303a6.htm https://www.cdc.gov/mmwr/preview/mmwrhtml/mm6303a6.htm [3]https://www.cia.gov/library/publications/the-world-factbook/geos/xx.html https://www.cia.gov/library/publications/the-world-factbook/...
- coinjobber 9y agoAmen. Uber's fatal accident rate is now 50x that of sober human driver's. At their current rate of driving, they can't have a _lower_ rate until 2028! Inexcusable failure of their technology to prevent a human death in the most basic collision avoidance scenario.
- setra 9y agoA fatal accident at 50x the rate of a sober human driver with a study size where N = 1.
- coinjobber 9y agoNo. 3 million miles of observation. You can get a pretty exact and conservative estimate with a bayesian poisson process model. I don't have the time to run the numbers right now, but my guess is the posterior estimate that Uber's fatal accident rate is higher than a human's is >90%, even if taking the human accident rate as a starting prior.
- buvanshak 9y agoGood riddance. But I think this is not enough to send a strong message to all those who want to put half baked tech out there in hopes of "disrupting" something.. Also, I really hope all self driving testing is suspended until there is sufficient legal and testing ground work before these things are allowed on roads...
- ineedasername 9y agoPrecisely. Disruption can be a wonderful force for change and innovation, but "minimum viable product" is an astronomically higher bar when injury & death are the default failure modes.
- sokoloff 9y agoIt might surprise you that we were testing (selectively) autonomous cars on the autobahn in Germany in 1991 as part of the Prometheus project. Our tech was nowhere near as good as what's available now and so we compensated by having attentive drivers hovering over the E-stop button. I don't think that stopping testing over a single crash is warranted. Maybe suspend it for a short time while you look into it, but if we stopped medicine every time a patient died in a trial, society would be worse off.
- buvanshak 9y agoI am not asking to stop it for all time. Just device proper tests that deem a self driving vehicle to be worthy/competent enough to be on the road with a normal average human being behind the wheel. And then have every single self driving vehicle to pass before it can be put on the road. Is that too much to ask? I am just asking for a driving test for self driving tech. Humans undergo driving tests. Why give machines a free pass? As we have seen here, having a human on stand by just won't cut it..
- carlmr 9y agoSo if the tech can pass a human driving test, is that ok? Because those are a very low bar in some countries (like the US).
- swang720 9y agoThis is interesting, but can we avoid posting links to articles behind paywalls? Here's one on the subject that isn't: http://www.chicagotribune.com/news/nationworld/ct-arizona-uber-testing-20180326-story.html http://www.chicagotribune.com/news/nationworld/ct-arizona-ub...
- samschooler 9y agoJust click the "web" link then click the WSJ link. If you have no cookies from them, it gets you around the paywall most of the time.
- stordoff 9y agoFWIW, complaints about paywalls are generally off-topic, though I agree, a non-paywalled source can be useful. https://news.ycombinator.com/newsfaq.html https://news.ycombinator.com/newsfaq.html
- ineedasername 9y agoI think courtesy should dictate that paywalled articles be accompanied by at least a brief description. Perhaps a change to the FAQ to recommend such?
- dang 9y agoIt's helpful to post better URLs, and we've changed the submission to that one from https://www.wsj.com/articles/arizona-governor-suspends-ubers-self-driving-cars-from-roads-1522113198 https://www.wsj.com/articles/arizona-governor-suspends-ubers.... Thanks! Please don't post complaints about paywalls, though: https://news.ycombinator.com/item?id=10178989 https://news.ycombinator.com/item?id=10178989. They're off-topic, and as long as standard workarounds exist (which users routinely post to these threads), paywalled articles are ok. Not that they aren't annoying; they suck. But HN would be quite a bit worse if all such articles were cleared out of here.
- owenversteeg 9y agoSaying nothing about Google/Waymo, it appears they're allowed to continue? Their self-driving technology seems far more advanced, I think it would be unfair to punish all the players for the bad actions of a few. In particular, Uber seems to have been operating self-driving cars recklessly and in an attempt to "move fast and break things". When those things are people's lives, that's unacceptable. I'm all for interesting technology, but sometimes existing rules are there for a reason. I particularly hope this doesn't dissuade individual states from making leaps to embrace a particular technology. I think that's one benefit of the US: individual states have a lot of power to allow a new technology or permit testing that's illegal or in a gray area in the rest of the country. Arizona made a bet on self-driving technology. I applaud Arizona, even if the bet in this case might have turned out badly.
- onetimemanytime 9y agoOr maybe Uber got unlucky and they had to be suspended. I wonder how many miles they had and every how many miles does an accident happens with human drivers.
- EpicEng 9y agoIt's been posted in related threads, bit many, many more human miles are driven per fatality. The problem is that the number of self driving car incidents is too low to reliably extrapolate out.
- Erlich_Bachman 9y agoWatch the video of the accident. Even though it's dark (llooks to have been manipulated afterwards), you can still judge the actual situations in terms of the position of the lights, and the car and pedestrian - to see that it was obviously the car's fault because it should have seen the pedestrian with the lidar, or if the lidar was deactivated - it should have driven with much lower velocity.
- ggg9990 9y agoUber did not get unlucky -- they are simply an evil company. Their cars need human intervention every 13 miles as opposed to Google's every 5,600 miles -- they were not fit for the road but Uber let them out anyways. Uber should be shut down and the principals should get life without parole.
- Game_Ender 9y agoI wonder how long the investigations are going to take, they usually take months to a year in these cases. Or have for Tesla at least. The political side is interesting to look at as well. The governor directly issued the order and he is up for election this year [0] and he played up the positive PR of the Uber and others coming to the state. 0 - https://en.m.wikipedia.org/wiki/Arizona_gubernatorial_election,_2018 https://en.m.wikipedia.org/wiki/Arizona_gubernatorial_electi...
- takeda 9y agoBased what I learned about Uber their self driving cars probably run on NodeJS. I'm only half joking.
- justicezyx 9y agoO.M.G.
- bryanrasmussen 9y agoI don't actually know what a self driving car should run on, I would want to make it in Erlang or Rust but - what do you think a self driving car should be running on?
- carlmr 9y agoFor real-time embedded there are basically 3 options, C, C++ and Rust. And Rust isn't that widely used, so my guess would be on C/C++. You can't have a GC.
- icebraining 9y agoWhat about Ada?
- talibandito 9y agoFrom what I know Ada is basically a legacy language mostly used by the military now a days.
- carlmr 9y agoTrue, I forgot about Ada. While ugly, Ada2012 is a very modern language with a lot of benefits compared to C, especially for embedded control systems programming. Ada got a bad rap in the past and didn't spread very wide outside US military circles. But it is a very good language. For adoption today it would need some big backing though.
- 9y ago
- johngalt 9y agoSeveral comments here making comparisons to human drivers. As if autonomous cars are ok so long as they are at or below parity with humans. This is statistically true but a complete misunderstanding of human nature. Human risk tolerance varies drastically depending on control/participation. What is the acceptable casualty rate of elevators?
- jMyles 9y ago> Several comments here making comparisons On one hand, you are (reasonably) calling out comments for questionable comparisons, but on the other hand... > What is the acceptable casualty rate of elevators? ...you've made one of your own. I haven't yet formed an opinion on the underlying question (of what kind of risk aversion is appropriate in the face of automation), but I do notice that it evokes questionable comparisons from all sides.
- mtgx 9y agoHis point was that machines should be held at much higher standards than humans. This is a point I'e been making since the beginning myself. If you want self-driving cars to be accepted, they need to be at least 10x better than the best human drivers. Comments such as "self-driving cars only need to be 50% better than humans on average" are absolute nonsense. Humans will never go for that.
- fastball 9y agoI don't think the risk of death from walking up the stairs is very high... so I'm assuming elevators are about equivalent, because people have definitely died in elevators.
- eftychis 9y agoActually, death by elevator is extremely unlikely. Comparatively staircases are death traps. Source by googling a bit (you can find many more): http://www.lifeinsurancequotes.org/additional-resources/deadly-statistics/ http://www.lifeinsurancequotes.org/additional-resources/dead... (1 in 10,440,000). That said I wouldn't trust an elevator in China or what you would consider underdeveloped countries (incorrectly called 3rd world) (I have heard various stories of cutting corners and lack of inspection). Self-driving cars don't need to be 10x better, but they are going to be. Humans introduce a delay of 1sec in best cases which is in the same order of magnitude to a car going 60 to 0. Again as mentioned above by others, self-driving cars don't experience fatigue, attention fatigue, emotions, focal vision degradation due to light conditions etc. Uber tried to cut corners from what is apparent [to me and quite a few others I guess]. I am really trying hard to imagine a scenario where the LiDAR would have missed the pedestrian. From my knowledge there are no stealth-material-dressed pedestrians out there, no matter how cool that idea would be for a movie. Try to check youtube about videos of the Uber car's behaviour in San Francisco and you would see issues a year ago. Driving conditions in Phoenix are much better (personal experience and I have witnessed self-driving cars in both cities). Uber needs to drastically improve their quality assurance and scenarios. (I am not trying to reach any conclusions and pass judgment without having all the evidence at hand. Even though my comments are extremely critical.) I think it is a good idea we Waymo, Uber, etc propose a standard for adoption with different levels and an independent committee argues and improves on it and we reach a common list of milestones. I think otherwise state governments are making decisions on a whim. Of course that path is a public relation nightmare as they will have to accept existing issues. Nobody wants to write down there is a 0.001% a pedestrian will die in this scenario, even if with a human driver behind the wheel the chance would be higher; for instance 1 or 10% or even 50%.
- kosei 9y ago> “Improving public safety has always been the emphasis of Arizona’s approach to autonomous vehicle testing, and my expectation is that public safety is also the top priority for all who operate this technology in the state of Arizona,” Mr. Ducey said in his letter. “The incident that took place on March 18 is an unquestionable failure to comply with this expectation.” This after the police claimed no fault.[1] Not quite sure how those two statements square... [1] http://fortune.com/2018/03/19/uber-self-driving-car-crash/ http://fortune.com/2018/03/19/uber-self-driving-car-crash/
- galdosdi 9y agoTwo different and unrelated agencies and levels of government with different incentives. The state has strong precedence over local authorities though, so whatever some garbage local keystone kops have to say is kind of irrelevant here. Local coppers are known in general to have a bias in favor of motorists for obvious reasons...
- shakna 9y agoThe investigation isn't complete, but Uber did disable the car's own safety features [0]. Both Volvo and Intel tested their software against the low-grade video that was released, and it was able to detect the impending accident and reduce the impact to where the pedestrian would likely have survived. [0] https://www.bloomberg.com/news/articles/2018-03-26/uber-disabled-volvo-suv-s-standard-safety-system-before-fatality https://www.bloomberg.com/news/articles/2018-03-26/uber-disa...
- icebraining 9y agoUber did disable the car's own safety features Well, of course they did. Can you imagine testing a self-driving system when another system is also controlling the car? What happens when they send contradictory commands? Race conditions are bad enough outside of actual cars!
- rimliu 9y agoThe should ditch the human out of the care too. What happens if it gives contradictory command? More seriously, as it was already pointed out elsewhere: Human > car safety features > AI. So it is pretty clear what to do when the car wants to engage emergency braking.
- paul7986 9y agoHow is this company still in business...karma is chasing it all the way down to oblivion!
- smnrchrds 9y agoOutside of Hacker News, I have never encountered anyone who followed, was fully aware of, or cared about Uber scandals. From my conversations, it seems that people in my city hate taxi companies so much that it takes more than a couple of manslaughters to make them stop using Uber.
- deleted 9y ago[deleted]
- peterept 9y agoI don't know much about the logic used in self-driving cars - but I do wonder how they will handle roundabouts. We have a lot of them here (Australia) and you have to give way to a car that is approaching or entering the roundabout on your right - which often is opposite you. How will a self-driving car cope with detecting a vehicle behind the concrete barrier of a roundabout?
- pfarnsworth 9y agoIn Mountain View, I once saw a Google/Waymo car trying to cross traffic and turn left onto Rengstorff Ave, which is a pretty busy street in the evenings. As a human driver, I wouldn't know how to do it unless there was a miraculous break in traffic, I would have just turned right and go the opposite direction and make a u-turn elsewhere. So like you, I'm curious how it handled that situation as well.
- kayoone 9y agoIn Europe we have a lot of roundabouts that work the opposite way, priority is given to anyone inside/exiting the roundabout, which would seem easier to handle for automated systems.
- Symbiote 9y agoI think that's what is meant, remember Australians drive on the left.
- rimliu 9y agoAlso in Europe it depends what road signs are at the roundabout. They have no special status in my country and can be looked as a "T" intersections connected to a circle. In most cases there is a yield sign for the cars approaching the roundabout so the cars already on it have right of the way, but it is not the case for all roundabouts. Also, some roundabout allow exit from the two rightmost lanes, others just from one—it also depends on signage.
- caf 9y agoNo, the roundabout rule is that a vehicle entering a roundabout must give way to any vehicle already on the roundabout, or a tram that is entering or approaching the roundabout. See the Australian Road Rules, Part 9: https://www.pcc.gov.au/uniform/Australian-Road-Rules-19March2018.pdf https://www.pcc.gov.au/uniform/Australian-Road-Rules-19March... If you are approaching the roundabout and someone enters it from your left before you get there, you have to give way to them.
- PeterStuer 9y agoUber and responsibility doesn't seem like a great cultural match.
- philip1209 9y agoThis seems like a lesson in why government regulation can be good.
- coryfklein 9y agoThe government regulation allowed the Uber on the road, so it may be a lesson in why government regulation doesn't work or provides a false sense of safety.
- throwaway2048 9y agoclearly the better alternative would have been no regulation, and uber continuing to operate (???)
- coryfklein 9y agoI just meant to point out that "regulation" on its own isn't inherently helpful. We shouldn't applaud "regulation" for its own sake, but should consider what the regulation actually is and how it functions when evaluating its value. Obviously our roads need some regulation, and in seems that the regulation in Arizona wasn't the right regulation.
- philip1209 9y agoI specifically mean that Arizona chose to allow self-driving cars on their roads with little regulation and that lack of regulation contributed to the death of a pedestrian.
- coryfklein 9y agoYea, I'm really splitting hairs at this point, but you can't use the Uber incident this month as evidence to support the claim that "regulation is good". You could use the incident as evidence that "Arizona's regulation is not good". We _can_ support the claim "regulation is good" by pointing to other states that did regulation right and seeing how their fatality rates are better than Arizona's. But yes, I agree with the spirit of the statement, Arizona needs better regulation, and it seems pretty clear that "regulation is good" in this case.