9 ms·
The Menace and the Promise of Autonomous Vehicles
- taneq 8y ago> What does it mean to experiment with technology that we know will kill people, even if it could save lives? This tagline ignores the fact that every new technology ever invented has killed people. Machines fail. Buildings collapse. Bras and vending machines and popcorn and fire hydrants all kill people. The simple passage of time on large projects kills people. People are going to die no matter what. If we can reduce the body count then that's a win.
- ArekDymalski 8y agoYour remark has led me to this article: https://www.standard.co.uk/news/death-by-bra-7225474.html https://www.standard.co.uk/news/death-by-bra-7225474.html Thank you :)
- buvanshak 8y agoSorry. Cars have a slightly more potential to kill than bras and vending machines and people dying of natural causes..
- Fricken 8y agoHeart disease, cancer and chronic lower respiratory diseases are the top 3 leading causes of death in the USA, all regarded to be natural causes. Accidents are #4, and car crashes are included in that.
- pimmen 8y agoAnd if we exclude the age group to the people who use cars the most, like people in middle age driving around their family? I would guess most of the people in that car are not that likely to die from 1, 2 or 3 anytime soon.
- Fricken 8y agoYeah, cars are best for killing (and crippling) people in the prime of their lives.
- letsgetphysITal 8y agoWe absolutely should stop research into chemo- and radio- therapies as cancer treatments because they may kill the patient. Is this article a satire?
- viburnum 8y agoAmerica's car death rate is far higher than most rich countries. We could save 20,000 lives per year just by following best practices, but nobody gives a shit, so let's all fantasize about magic cars.
- a_imho 8y agoAgreed, autonomous vehicles might offer some benefits inclding road safety in the very long run, however we could do so much better right now if laws were enforced / people were actually following them or just care. The system is designed with plenty of buffer in it, it is just the nature of most drivers to disregard speeding limtis and safe trailing distances. As a consequence imo it is misguided to measure AVs against average drivers, because more average drivers on the road is exactly what I would rather not have.
- KKKKkkkk1 8y agoThe article is claiming that the development of self-driving cars will require killing people on public roads. That's a tantalizing prospect ("kill thousands now to save millions later"), but no evidence or arguments are presented why. Sure, there will be deaths due to road accidents, but is it really inevitable that more people will die due to failures of engineering like Uber's or Tesla's?
- dbasedweeb 8y agoUtopias are always justified this way, and the worst crimes are committed as a result. After all, if a potentially limitless number of lives are saved in the future, you can argue for sacrificing millions today. Somehow people never seem to understand that nothing real or worthwhile is down that road, and as a species we go down it repeatedly. I think the best argument against this, is to demand that the person willing to let many die is the first to die.
- maxxxxx 8y ago"I think the best argument against this, is to demand that the person willing to let many die is the first to die. " That's why I am saying that all road testing of autonomous cars should take place in the neighborhoods where the executives and their children live and are on the road. That would align the incentives of the company and the public.
- Animats 8y agoI stopped reading at "Central to AV testing is the “trolley question”. No, that's not a central problem. If it was, it would be covered in driver's ed and driver testing. It's not. Most accident avoidance is pure braking. On the evidence, the central question in AV testing is "can we trust the human to take over fast enough when the automation does something bad?" The answer appears to be "no". He also parrots Uber's whining about California's "regulation" of AV testing. The California DMV autonomous vehicle testing rules mirror those for human driver learner's permits. You have to get a license. You have to show proof of insurance. You have to report accidents. You have to have a licensed driver on board. DMV can pull your license if you have accidents. You can't carry passengers for hire. You can't drive a heavy truck. No surprises there. DMV does require reports of minor accidents and of disconnects, and they publish that data. That's what irks the companies with bad automatic driving systems - public disclosure that they suck.
- joe_the_user 8y agoNo, that's not a central problem. If it was, it would be covered in driver's ed and driver testing. It's not. Most accident avoidance is pure braking. Absolutely. This sort of hypothetical-spinning kind of mirrors the way concerns about general AI "taking over" mask the problem of dumb AI making disturbingly biased decisions and putting us at the risk through the failures of fragile logic. Autonomous vehicles promise all sorts of paradoxical effects, haven't been demonstrated to be more reliable than humans, etc. A bit of focus on the real near-future could benefit these speculators.
- paganel 8y ago> It's not. Most accident avoidance is pure braking. This is false. "Accident avoidance" also means being constantly aware of your surroundings and slowing the car down before an incident (like a person suddenly getting in front of your car) might happen. This strategy was actually posed as a question on my driver's exam test: What should you do if you see a bunch of kids playing on the sidewalk close to the road? Correct answer: You should slow down, because the kids might suddenly hop in front of you. The same goes for bicyclists.
- 8y ago
- RcouF1uZ4gsC 8y agoOne of the most dangerous states of mind, is to be pursuing something that you are convinced will save people's lives in the future, but that might hurt people now. This type of thinking is unrivaled in causing previously normal people to do horrendously bad stuff. Add to this the potential for profits, and it becomes a very toxic mix, where a person can be killing people while trying to make a profit while fully convinced that they are a very good person and are doing humanity a great service. We have historical experience with an analogous situation. Medical experiments and drug development. People became so focused on "the greater good for humanity" that they did horrible stuff to study participants (see Tuskegee Syphilis Study). Or drugs were not adequately assessed for safety in pregnant patients and severe birth defects occurred (see thalidomide). This is one of the reasons we have Institutional Review Boards to review medical studies and the FDA to review drugs, because people involved in a project like this are so good at fooling themselves. When developing drugs, pharmacy companies aren't allowed to say "This drug will cure X and benefit so many people. We would like to ignore the side effects happening now because the cure will be so beneficial in the future." We need the equivalent of an Institutional Review Board/FDA for self driving cars. As part of the submission, the engineers have to document what sensors they are using, what the limitations of those sensors are, what algorithms/machine learning techniques they are using, and the limitations of those. They also need to disclose the engineering tradeoffs and design decisions (for example, not having LIDAR). Then there need to be tests for self-driving cars for safety on the unhappy path. What happens if there is a concrete barrier in front of you? What happens if the lane markers are bad? What happens if the car in front of you brakes? What happens if you are driving with the sun facing you? What happens if the truck in front of you drops something in the road? What happens if a child darts in front of you? And so on... Then based on the testing, certify the car for self-driving only in certain conditions that were shown to be safe based on the independent testing. After a self driving car is on the road, all data collected by the self driving cars should be shared with the review board so that there can be ongoing monitoring. In addition, users of those cars should be able to report incidents and near misses to the review board.
- fraudsyndrome 8y agoYou know, I'm not in this field but this should be stuff that I assume would/should already be done BEFORE even putting it out to the market.
- paul7986 8y agoNot paying thru the nose to be a Musk guinea pig nor going to walk the streets of Pittsburgh .. avoiding having an Uber car run me over. Progress will be a killer!
- jadedhacker 8y agoIt would be much easier to countenance arguments that self driving cars will save us all if humans weren't such good drivers. http://www.iihs.org/iihs/topics/t/general-statistics/fatalityfacts/state-by-state-overview http://www.iihs.org/iihs/topics/t/general-statistics/fatalit... In Massachussetts, the number of deaths per 100 million miles driven is 0.66. Those are the massholes. By contrast, it was reported that Uber required human intervention once in thirteen miles while Waymo required human intervention once every 5000+. Those are not directly comparable to fatality statistics because the interventions are also for things like avoiding disturbing events like hard braking, but they are still illustrative. https://arstechnica.com/cars/2018/03/leaked-data-suggests-uber-self-driving-car-program-years-behind-waymo/ https://arstechnica.com/cars/2018/03/leaked-data-suggests-ub... Let those orders of magnitude sink in. The humans are driving in all weather conditions all over the country while self-driving cars are in (generally) sunny relatively controlled areas. Personally, I have hope that we can eventually beat humans, but these kinds of statistics call for a careful, deliberative introduction of this technology.
- TangoTrotFox 8y agoSo you think it's reasonable to compare the rates that humans not only get into crashes, but actually die (or kill) in crashes - to the rates that a human takes over for an early alpha tech version of self driving vehicles? Your stats not even remotely "illustrative" of anything other than what they say of the presenter.
- jadedhacker 8y agoThis would be academic if someone hadn't actually died. I don't know the numbers, but I'm fairly sure that just tanked the self-driving stats pretty hard. EDIT: Also, consider what avoiding "hard braking", which was a large reason for human interventions iirc, actually means. It means either that the computer jams on the brakes hard for no reason or that it only realized it needed to brake substantially after a human would have realized the necessity. EDIT2: Uber had driven 3 million miles at the time of the death according to the below article. A single incident does not statistics make, but based on the NSTB analysis so far, that version of the software made a systematic error. That puts Uber's score at very handwavy 33.33 per 100 million miles. I'm sure the number will change with software revisions and just from random chance. https://www.nytimes.com/2018/03/19/technology/uber-driverless-fatality.html https://www.nytimes.com/2018/03/19/technology/uber-driverles... EDIT3: I should mention that I was initially hesitant to compute this "statistic" because it's so raw and will wildly fluctuate. Interventions are more solid statistically. I don't think it's crazy for someone to look at a heavy box of metal that requires constant supervision to do the right thing and conclude in the absence of (appropriately) extensive testing that it's dangerous.
- cyle_vogt 8y agoWorth reading: https://www.theinformation.com/articles/inside-cruises-bumpy-ride-the-limits-of-self-driving-cars https://www.theinformation.com/articles/inside-cruises-bumpy...
- rob_raviolli 8y agoI recently read this paper (http://sjha8.web.engr.illinois.edu/resources/DSN2018.pdf http://sjha8.web.engr.illinois.edu/resources/DSN2018.pdf) from the dependable systems conference. Seems like AVs are no where near ready. This is not just a Uber problem - it is pervasive across most manufacturers. The rate at which these AVs are getting better essentially means we won't have "human level" cars in a long time.
- rubidium 8y agogreat reference, thanks for sharing. This type of systems level analysis is what I've been looking for (and I think what all the av hype is ignoring).
- icebraining 8y agoI have no idea if AVs are near ready, but that analysis seems a bit shallow. All the dataset gives them is the number of disengagements and the miles driven each month, but not all miles are necessarily the same; for example, it makes sense that Waymo et all would start taking their cars to streets with more traffic/people/etc as they get more confident in the system's ability to navigate in easier areas.
- Fricken 8y agoOver $100 billion has been invested in this as yet unproven technology, there is no historical analogue for that. This money has been invested either because the players involved stand to make a lot of money, or else they stand to lose a lot of money if they don't invest in the technology. Moreso the latter than the former, really. Plenty of lipservice is paid to the potential safety benefits, along with a laundry list of other social benefits, but really it's about money. By logical extension, autonomous vehicles have to be safer than human drivers, or they just won't fly. Too many accidents invites regulatory scrutiny, time consuming safety audits, and worst of all, public backlash. If you look at the players across the industry they can be broken into two distinct categories: the offensive players, and the defensive players. The offensive players: Namely Waymo, but also Cruise, Aurora innovation, Drive.ai, Argo.ai, Zoox and Nutonomy: their safety records are impeccable. But there are two prominent defensive players who both would be better off if AVs were never a thing: Tesla and Uber. For them, screwing it up for everyone is a net benefit to their business models. Now, if we were actually concerned about safety, we wouldn't be driving, we'd be riding bicycles (or walking or taking the train/bus), and our cities would more closely resemble what they were in the 1930s or earlier. But there's no money in bikes, and everybody hates them.
- deleted 8y ago[deleted]
- tim333 8y ago> It will take billions of miles—and some unknown number of people killed—to gauge whether, by a statistically significant margin, AVs are safer than human-driven cars. This kind of lumps all the AVs together. There is evidence though that Waymos are already safer than human divers - from 2012: > Google announced Tuesday that its self-driving cars have completed 300,000 miles of test-drives, under a "wide range of conditions," all without any kind of accident. (The project has seen a few accidents in the past — but only with humans at the wheel.) > To put that into perspective, the average U.S. driver has one accident roughly every 165,000 miles. https://mashable.com/2012/08/07/google-driverless-cars-safer-than-you/?europe=true#VYLbEm9KDgqB https://mashable.com/2012/08/07/google-driverless-cars-safer... Since then I think they have logged about 6 million miles and had one semi at fault accident when their car thought a bus would give way but it didn't and hit them. I don't think Waymo would been looking to order 62,000 vans if they were not confident they would be safer than human drivers. They have also had far more accidents with humans at fault crashing into stationary Waymos than the other way around. On the other hand Ubers crash seems like a reckless effort to try to catch up by cutting corners and Tesla's it's your fault if you didn't look for a few seconds attitude seems iffy.