6 ms·
This is yet another instance of the "what is the acceptable risk for introducing a new technology" problem. New technologies tend to be rather unsafe when intr
by inglor_cz 5y ago
This is yet another instance of the "what is the acceptable risk for introducing a new technology" problem.
New technologies tend to be rather unsafe when introduced: steam engines, cars, trains, airplanes, electricity, gas heating, all those things tended to kill the early adopters and some bystanders as well. Then they improve and the next generations get used to them and their inherent risks that diminish over time but rarely get down to pure 0.
We are much more risk-aware than our ancestors, though, because we no longer live in a world of ubiquitous premature death. With an average life expectancy exceeding 80 in many countries, we tend to be much more careful than our ancestors in times when life expectancy was 47. (As in the USA in the year 1900.)
I can understand that - I do not want to die under wheels of a robot gone wild any more than anyone else - but we should still find a balance, unless we want to stagnate indefinitely.
The risks of self driving cars are nothing compared to the risks of biotech, which is just coming of age. But biotech holds a lot of promises, too.
PS.: Frankly, my worst fear about self-driving cars is malware and ransomware, not honest mistakes in the code. Computer security is basically an oxymoron and some people are smart and evil at the same time.
- jdavis703 5y agoI have no problem driving or biking among the LIDAR-based systems from Waymo and others in the SF Bay Area. I’ve seen them do stupid stuff, but it was always on the side of safety. On the other hand I’ve seen hands-free Tesla drivers reading books, using their phone, etc, which is both stupid and dangerous.
- theshrike79 5y agoTesla definitely should have a camera watching the driver and detect if they're being an ass. if user.is_an_ass == true: disable_autopilot(days=7) That should teach them pretty quickly.
- panick21_ 5y agoThey literally do have that.
- theshrike79 5y agoIf they have that, why then do we see videos and news of people reading newspapers and sleeping behind the wheel of a Tesla? The only feature I've heard is that if you don't engage with the AP by applying torque to the steering wheel it'll disable it until you come to a full stop and park the car.
- avereveard 5y agowait were these 5 years of hype all for a self driving car that I've to constantly be driving?
- tgv 5y agoSelf-driving cars can wait. Nothing of importance depends on it. But if it's regulated properly, and liability is where it should be, then introduction can come, in due time. Biotech is indeed a whole different can of worms. > not honest mistakes in the code It's not mistakes in the code, but oversight in the design that worries me.
- inglor_cz 5y ago"Nothing of importance" I would argue that the total amount of time spent driving is huge and that as humans, with limited lifespans, we could mostly use that time for better purpose. (Not necessarily for work. Even Netflix would be better.) I personally live in a country where being carless is feasible, and given that I hate driving, I am indeed carless. But I feel sorry for anyone in my situation who really does not have much choice and must spend X hours weekly behind the wheel.
- rjzzleep 5y agoOh you mean a thing you could solve by public transport and proper street planning? Or reducing commute due to remote working in non physical labour? I feel that there are a lot of simpler ways to address the issues at hand.
- inglor_cz 5y agoIs rebuilding the U.S. of 2021 to a totally different modal split really simpler than self driving cars? Even California's high speed rail seems to be languishing in limbo.
- zsmi 5y agoSimpler in concept maybe. In practice, I am not so sure. Just one example: In SV we voted for a tax increase in 2000 for a Bart Extension to San Jose. If we're (really) lucky, we might get it in 2030 after spending ~$10B. The total distance is ~20mi with less than 10 stations. https://en.wikipedia.org/wiki/Silicon_Valley_BART_extension https://en.wikipedia.org/wiki/Silicon_Valley_BART_extension
- hiddencost 5y agoNo it's not? It's about whether a specific company who has an extensive history of irresponsibility and killing people should be allowed to test. Tesla is far behind the competition and taking extensive risks to catch up. There are companies who are much further along and take safety seriously.
- loosescrews 5y agoExactly. This is a problem with Tesla, not autonomous vehicles. There are lots of other autonomous vehicles being tested in San Francisco. In fact, it is a pretty popular place to test autonomous vehicles. Waymo is even offering a self-driving taxi service in San Francisco: https://arstechnica.com/gadgets/2021/08/waymo-expands-to-san-francisco-with-public-self-driving-test/ https://arstechnica.com/gadgets/2021/08/waymo-expands-to-san...
- kube-system 5y agoThis is less about the introduction of self driving cars as a new technology, and more about a company introducing a level 2 driving assistance feature and calling it “self driving”, when in technical terms, it never takes over driving responsibilities from the human driver. The danger in this case is entirely manufactured. Level 2 driving assistance features are safe when they are not oversold.
- gitfan86 5y agoThe Tesla FSD Beta program requires drivers to understand this is a level 2 system. The human driver has to be in the driver's seat paying attention 100% of the time. If they move to the back seat they will be banned from the system.
- xattt 5y agoYou don’t have to move to the back seat forgo paying attention in a vehicle.
- toast0 5y agoSo the Full in full self driving is like the Full in full speed USB? Actually it means not full, but mostly empty?
- gitfan86 5y agoBeta as in it isn't done yet, but when it is done it will be safer than most drivers.
- kube-system 5y agoThe SFCTA isn't raising concerns about hypothetical technologies that Tesla might release at some point. They are raising concerns about what is actually being delivered in reality.
- kube-system 5y agoSafety critical systems should never be labelled with conflicting statements regarding safe use. Disclaimers aren't a safe way to mitigate this. When given conflicting statements, people tend to psychologically cling to the statement that was the most strong, prominent, or reenforcing to their predisposed views. Also, the driver of a car is not necessarily the owner. There is a long history of people dying because they were confused by conflicting information about safety systems meant to protect them. Safety engineering starts with consistent and clear messaging. It's not 2012 anymore, and there are now a number of vehicles with level 2 driving assistance features on the market. Yet, Tesla is the only manufacturer consistently in the headlines when one of their users is peer-pressured into showing off features their car doesn't actually have.
- passivate 5y ago> but we should still find a balance, unless we want to stagnate indefinitely. I don't think there is any risk of stagnation. Maybe we should work on why we need cars in the first place. We can simultaneously work on public transport, or virtual presence, etc.
- inglor_cz 5y ago"Self driving cars" are just a subset of "autonomous vehicles". Public transport would benefit greatly from driverless buses. Bus drivers are a scarce resource. Where I live, there is a shortage of people willing to rise at 4 a.m., take responsibility for 40-50 lives at a time and, at the same time, have unpleasant interactions with members of the public (meeting a few asshole passengers everyday belongs to the job, unfortunately).
- passivate 5y agoYou're right. I guess I'm thinking that mass transport systems with dedicated/fixed routes can be designed in ways that allow for much safer operation, versus an open-ended system like FSD. e.g. no risk of pedestrians on train lines.
- rhinoceraptor 5y agoIt just seems like this whole endeavor is solving the wrong problem. Building a self driving car is like making a faster horse. Individual cars on roads is incredibly inefficient and even EVs have a huge negative impact on health and air quality from tire and brake dust.
- phalangion 5y agoBut it’s a problem that can be addressed at the individual level, rather than requiring large societal shifts like a move to more public transportation would. As much as I love the idea of easy-to-use public transport, the changes in US society needed to make that happen are unlikely to come in the next 5-10 years.
- foobiekr 5y agoI think it should be obvious that the FSD thing is and has always been an attempt to boost Tesla's appeal and stock valuation. Right now it really seems like the company is playing chicken with regulators so that they have a plausible target of blame and an excuse for not releasing a feature they claimed was almost ready _years ago_ and sold to people which is quite obviously not even close to ready. It's a pretty common strategy in scams.
- GavinMcG 5y ago> life expectancy was 47 Life expectancy for a child. A white person reaching the age of twenty would have an average life expectancy into their 60s. Which is still lower than today, of course, but I'm not convinced that the difference supports a conclusion about differing attitudes toward risk.
- inglor_cz 5y agoUnless they were killed on the battlefield ... this kind of risk seems to have gone down too in developed states, and that is pretty significant. Europe in the first half of the 20th century, with exception of a few lucky countries, was one big war graveyard.
- entee 5y agoI'm not sure what you're arguing in terms of acceptable risk. Biotech is incredibly regulated, specifically because the risks are so high, effectively there is very little acceptable risk. In biotech, a patient dying due to your drug is a Big Problem that will at best cause you to put a disclaimer on the package (see Black Box Warning) and at worst immediately end your drug's prospects. We can argue about trade-offs (if you've got terminal cancer, maybe a rare heart event is a worthwhile risk, probably less so if you have a rash), but this is exactly the way it should be. Self driving cars are a nice luxury, especially in city driving, not something that radically improves our world. You get to read your phone instead of paying attention, and the trade-off is someone might get killed. It's like treating a rash with a drug that could give you a heart attack. That's a far cry from, "with this technology something that took days and $$$$ now takes hours and $" as was the case with all the older examples you listed. If self driving cars were more like airplanes, I'd have a little more faith. Tesla's marketing BS doesn't inspire me with lots of faith. On black boxes: https://health.clevelandclinic.org/what-does-it-mean-if-my-medication-has-a-black-box-warning/ https://health.clevelandclinic.org/what-does-it-mean-if-my-m...
- nexuist 5y ago> You get to read your phone instead of paying attention, and the trade-off is someone might get killed. You get to drive, and the trade-off is someone might get killed. Your comment almost makes me think you haven't driven a car before, because you would remember the dull terror of seeing your life flash before your eyes for the 40th time this year because some moron ran a red and slammed the brakes in the middle of the intersection you were about to cross. Until recently motor vehicle accidents were a leading cause of death in the US. Saying that self driving would just be a luxury feature is truly a luxury position compared to those that have lost loved ones to drunk driving, speeding, snow, rain, new drivers, old drivers, blind drivers, and any other of the myriad of ways to get yourself killed on a road. All of which would disappear with level 5 self driving. > That's a far cry from, "with this technology something that took days and $$$$ now takes hours and $" Extrapolate the future and realize that once self driving is solved for one vehicle it's solved for all of them, and truck/bus/taxi driving as a profession will go bust. Without having to pay human drivers that also need breaks, pensions, health insurance etc. all these services can offer lower prices.
- phkahler 5y ago>> This is yet another instance of the "what is the acceptable risk for introducing a new technology" problem. That's a strawman. Just make the company fully liable for any injuries or deaths from accidents involving their cars. The market will figure it out from there. But that's not how it's being handled at all.
- esteth 5y agoAt level 5 that makes sense - there is nobody in the vehicle responsible for the safety of the vehicle.
- FireBeyond 5y agoTesla in legal disclaimers: "It's all on you, driver." Tesla in FSD advertising of current state, not future goal: "The driver is only there for legal reasons. The car is driving itself". To be clear, that's a word for word quote from previous Tesla marketing (because I'm sure someone will chime in to say "Tesla doesn't do any advertising!").
- sidibe 5y agoFYI that video is still the main feature on tesla.com/autopilot
- dilyevsky 5y agoNot sure the math works out here. Even if you have 100 deaths/y nationally assuming wide scale rollout (tens of millions of cars, billions of trips) which would be astonishing achievement that’s still $ billions of payouts every year (assuming ~10M payout per death). Yeah… not gonna happen
- amelius 5y agoThe real risk with self-driving cars is managers rushing automatic software updates when bugs are discovered.
- bumby 5y agoSchedule and cost risk are always going to be present. I don’t know how you’ll fully mitigate that given humans struggle with adequately gauging risk and asymmetrical incentives in the face of low-probability events.
- notsureaboutpg 5y agoI think stagnating is okay. Think about medical science. Drug innovation at the rapid pace of the current year pulls out life saving medicines every year only to keep them out of reach of the vast majority of patients due to price. Technically more lives are saved this way, but the emotional toll on people of knowing a medicine exists to save their life but they will not be able to afford it is high and not measurable in statistics. Perhaps it is not worth the rapid innovation cycle. If the costs are so high and can only be recouped with extremely high medicine costs, maybe the pace of innovation should slow for the betterment of society.
- GhostVII 5y agoWe could also introduce safety features that use the FSD hardware and software, but only to avoid accidents rather than drive the car. Once you have a system that you know does a better job than humans at avoiding accidents, it is safe to introduce a system that actually drives the car.
- bumby 5y agoThis is, to a certain extent, a post advocating for regulation. Take the aviation example. It was certainly dangerous for early adopters when it was a nascent technology. But one of the reasons why the current level of aviation is relatively safe (they are one of the very few examples of five sigma quality) is because they are heavily regulated. Everything from maintenance to licensure to duty cycles are regulated. I think one of the risks of the current paradigm is the inability of US government to effectively introduce new regulation legislature. The new method seems to be to put the onus on the industry to regulate themselves in lieu of having to create actual legislation, which can lead to perverse incentives. In the instances where this isn’t the case, we have to deal with regulatory capture or the revolving door of industry/government which creates its own skewed incentives.
- odessacubbage 5y agoi wonder about the commercial feasibility of restricting auto pilot to specified zones, maybe even specified expressways similar to turnpikes. would this allow self driving technology the room to develop with a socially acceptable level of risk or would it condemn it to go the way of the concord?