15 ms·

I think the problem we have with self-driving cars is more social than technological at this point. Consider the hypothetical: a million self-driving cars on t

by mabbo 4y ago

I think the problem we have with self-driving cars is more social than technological at this point. Consider the hypothetical: a million self-driving cars on the road that, collectively, will have 1/10th of the fatal accidents that human drivers would have[0]. But, the ones they do have are accidents a human driver would almost certainly have avoided. Is this something we would accept? My guess is that no, we wouldn't. Because the accidents avoided don't make the news, but the accidents that occur- especially ones that you say "my god, how did it screw that up?" will make the news, and our perception would be that they are more dangerous. Until Waymo's cars are better than most humans in every single situation, they won't be able to win over the public perception war. [0]I'm making those numbers up. I acknowledge that. But it's a hypothetical so give me some leeway on this!

just-ok 4y ago
I agree with you entirely. Continuing with your hypothetical, even though we’d be 90% safer as a collective, the safety of the individual feels compromised: the risk of an accident is non-uniform when involving humans (depending on e.g. age, experience, safety, alertness, etc.), but becomes uniform (or at least more uniform) with an algorithm in charge. That’s a tough thing for people to buy into.
- _cs2017_ 4y ago
  A very good observation. Based on your comment, I think we can relax the requirement stated by OP by saying: "Until Waymo's cars reduce any individual's chance of an accident." So for example, suppose a Waymo car is better than humans overall, but tends to do worse than humans when there's a small bump on the road. And suppose that all humans (in a given regulator's area, e.g., California) tend to encounter such bumps at roughly the same rate (per mile driven) over their lifetime. In that case, it's probably going to be acceptable, since every individual is better off. I don't know, maybe this is not impactful / obvious enough for people to care about? What certainly is obvious is that the safest drivers are much safer than an average driver (does anyone know of a study that estimates this ratio?). Therefore, at the very least, the threshold for Waymo should be not the average accident rate, but the accident rate for the safest drivers.
- jsbg 4y ago
  > the safety of the individual feels compromised Exactly. I have had zero accidents in 20 years; I'm not interested in a car that will lower the overall accident rate if it increases mine.
  - mertd 4y ago
    Even the most perfect defensive driving won't fully mitigate the unsafe driving of others though. I'd rather have them to be in autonomous vehicles.
    - xboxnolifes 4y ago
      Sure, but nobody thinks they are the bad drivers.
    - jjav 4y ago
      > I'd rather have them to be in autonomous vehicles. Who is "them" and who allocates the groups? We can't even get any kind of car control skill testing or training to be part of a driver license issuance.
alphabetting 4y ago
If 10% of the time they got into accidents humans would have avoided we wouldn't be where we are today. I can't imagine any scenarios where these cars get in accidents that a human would have certainly avoided. You also say avoided accidents don't make the news but I'm pretty sure footage of them avoiding accidents that humans would have no chance of will be a major part of their marketing.
- RhodesianHunter 4y ago
  >I can't imagine any scenarios where these cars get in accidents that a human would have certainly avoided. Then you've not been following the space. The one that immediately comes into mind is the Tesla that slammed into the side of the semi truck because it was painted blue like the sky.
  - alphabetting 4y ago
    I've been following it pretty closely. I don't consider Teslas to be anywhere near self driving. By saying "these cars" I was referring to Waymo.
- jjav 4y ago
  > I can't imagine any scenarios where these cars get in accidents that a human would have certainly avoided. If you follow the events with self-driving accidents, most of them are nonsensical crashes that no human would ever have done.
  - alphabetting 4y ago
    I follow AV news very closely. I'm talking about Waymo. The small set of accidents they've been accidents in have been limited to fault of other drivers with the vast majority being rear endings
alexose 4y ago
This is one reason I never understood Tesla's vision-based approach. In order to be accepted, self-driving cars don't need to be just somewhat better than humans most of the time. They need to vastly better in every situation, as you mention, to the point that they'll need every sensory advantage they can get. I got out of a ticket once because I didn't see a "no through traffic" sign against a bright sunset. No chance that same cop gives a self-driving car a pass, nor should he.
- szundi 4y ago
  Karpathy claimed they worked for years and could not reap the benefits from multiple sensors however hard they tried. He seemed really convinced and does not get to me as one who tells stuff just to justify cost reductions, like Elon sometimes is carried away.
  - muglug 4y ago
    It’s also possible they weren’t able to see benefits given the processors that were available to them at the time. It’s also possible that they were so far behind Waymo in the journey to FSD that they weren’t yet at a point where multiple sensors would make a significant difference.
    - alexose 4y ago
      This is my read on it as well. I listened to the same interview (it was definitely on the front page, if not at #1 for a while). I stepped away feeling like Karpathy had described a lot of good business reasons for not using other sensors, but not a lot of good technical reasons. Sensor fusion is hard, yes, but maybe not harder than perfectly re-projecting 2D pixel images into 3D vector space. Just my interpretation, but it felt to me like a hail mary because they were fully committed to being the first mover. Waiting around for LiDAR prices to come down would have meant that Waymo would have beat them.
  - impulser_ 4y ago
    I always thought it came down to two reason, costs and looks. Telsa has to sell a car people want to drive daily. They can't have a bunch of lidar sensors on their cars no one would buy them even if they could drive themselves. Also they would most likely cost a lot more due to lidar sensors not being cheap compared to normal cameras. Unlike Waymo who doesn't care about selling cars to people to drive daily. No one going to care that the taxi they are taking looks ugly as long as it get them to the place they are going for cheaper. The cost is also a less of a factor due to them being able to produce an income by charging people to ride in them.
gitfan86 4y ago
It is a valid point but the financial incentives are so big that some jurisdictions will allow it. In fact they already do allow these autonomous systems on public roads. That is going to continue to expand and since the financial incentives are huge even when deaths happen the governments will continue to allow it. And in fact some regulators fully understand the tradeoffs and will prefer autonomy for the better good of the public. An example of this is the Boeing 737 Max, those crashes wouldn't have happened if there were no autopilot systems. But regulators are not suggesting that all autonomous systems on planes be turned off because of the safety and financial advantages of keeping them in place even though they are obviously not perfect.
- pauljurczak 4y ago
  > Boeing 737 Max, those crashes wouldn't have happened if there were no autopilot systems. Bad example. MCAS was an obvious case of a criminal corporate behavior, not a tradeoff between overall safety vs. technical perfection.
  - gitfan86 4y ago
    It doesn't matter, the point is that relying on autopilot will open up crash risks that do not exist with manual flight controls
    - pauljurczak 4y ago
      It does matter! I'm assuming that any sane person, especially a highly trained professional, is not fully relying on automation and has at least some idea of its limitations. In the case of MCAS, corporate fraud distorted the level of trust pilots had in automation, leading to loss of life.
yyyk 4y ago
I agree it's a social problem, but IMHO it's a rather different social problem: Current cars, roads and car culture is adapted to human drivers, and AI is expected to be able to integrate into that. What if we would make cars and rules that are adapted to AI cars and ignore human drivers? e.g. Ban human drivers from some roads, allow AI cars with designs that exploit AI advantages (e.g. much better reaction time) but do not require or even allow human backup (enabling us to put the passengers in a secured shell), etc. I suspect we could than reach a 1/20th rate today.
lynx23 4y ago
You're hitting the nail spot on with this one for me. As a blind pedestrian, I very much feel I am in danger of falling into exactly that group of potential victims you are hinting at. Right now, I have the illusive comfort of the "Vertrauensgrundsaatz" which basically tells every driver obtaining a drivers licence that they need to take special care when it comes to disabled pedestrians. Sure, one might say these new self driving systems will "just" have to follow that same rule as well, but I am very much doubtful this is technically possible. So currently, I feel like the drive to put innovation on the streets and pull money out of pockets is actually actively endangering me in the future. Not very bright outlook I must say.
- nicbou 4y ago
  By your use of German, I assume that you experience a far higher driving standard than people in North America and frankly most of the world, and even that is far from perfect. The Vertrauensgrundsatz does not account for distracted, tired and inebriated drivers.
  - deleted 4y ago
    [deleted]
- jjeaff 4y ago
  My guess is that self driving cars are already far, far better than human drivers at not hitting pedestrians.
  - krferriter 4y ago
    My guess is that it depends a lot on the driver. There are a lot of really reckless drivers, or really bad drivers out there on the roads right now. I know people who've been in several (relatively minor) car crashes, driving the same-ish sort of route to/from college and work that I was during the same period. I think they're just a bad driver. I feel like they're driving recklessly when I'm a passenger in a car they're driving. So a self-driving car might be a better driver than them, while also being worse than the median driver, and much much worse than like the upper portion of the human driver spectrum, like the 1st quintile of the 20% of human drivers who are pretty good. Given how reckless some drivers are, I'd imagine the distribution of crashes among drivers is very skewed. I don't know numbers, but I'd expect to see something like 80% of car crashes involving (or objectively caused by) 20% of the drivers, with most car crashes involving people who have been involved in numerous crashes, and a large minority of drivers having been involved in zero car crashes.
GaryNumanVevo 4y ago
To me, self-driving seems like a band-aid fix for terrible car infrastructure. Driving a car is already one of the most dangerous things the average American can do. Due in part to larger, heavier vehicles, high speeds in residential areas, etc. Even if you had a "perfect" driver that doesn't prevent someone from ramming into you.
- rootusrootus 4y ago
  > one of the most dangerous things the average American can do About comparable to the risk of falling. Lower risk than suicide. Or death by opioid overdose. And of course, the most dangerous thing most Americans do, by a huge huge huge margin, is overeat and lounge on the couch.
- Supercoolfaggot 4y ago
  Imagine thinking a picture of a penis is sexual harassment...
larusso 4y ago
As a software engineer myself they will always lose the argument. I saw so many smart systems falter in some weird way that I would never trust a software system completely. I drive an EV with some AI based system that automatically throttles the car etc. But to trust my life and my Familie in the hands of this system (or any other) no thank you.
taeric 4y ago
To double down further on the social side of this, public transit is largely getting there faster than point to point driving. In that many trains and such are already largely "hands off the wheels" for operation. Relatedly, another "problem" with "self-driving" cars is that we want all of the convenience and ease of use, without adjusting liability and ownership considerations. Consider, if Waymo gets to the point where they have a self driving car that you have to have a subscription to use, do you own the car? Are you the liable for any accidents it has? To lean in on that hypothetical. I'd imagine a lot of families will use self driving cars to send kids to school. Is effectively a bus that terminates at your house. Who is liable for a mistake if the operation of it is completely remote?
- amf12 4y ago
  > Consider, if Waymo gets to the point where they have a self driving car that you have to have a subscription to use, do you own the car? Are you the liable for any accidents it has? I'd imagine the company would take on liability, as long as humans can't drive the vehicle or they aren't driving when the accident occurred. Mercedes already got the ball rolling on this [1]. [1] https://www.roadandtrack.com/news/a39481699/what-happens-if-mercedes-drivepilot-causes-a-crash/ https://www.roadandtrack.com/news/a39481699/what-happens-if-...
  - taeric 4y ago
    I can't see that path getting taken without you basically losing ownership of the car?
    - amf12 4y ago
      Agreed, but I can see it being solely a "lease" model. For companies to take on liability, they'd have to make sure a customer doesn't modify the vehicle and keeps up with the maintenance. They might also want the authority to disable "self driving" on non-compliant vehicles.
- rootusrootus 4y ago
  > public transit is largely getting there faster than point to point driving Aside from some niche cases in very dense cities, is this generally true anywhere? I've visited a lot of cities with various levels of public transit, and I can't think of many where it was faster. More convenient sometimes, sure, cheaper, yep, but faster? Not often.
10x_contrarian 4y ago
I think you're mistaken. There are still massive technological challenges. I have seen nearly no evidence that current self-driving car technology is even remotely close to matching the ability of a novice human driver. Sure while the "don't crash into things" algorithms may generally be fine, these systems seem to frequently deadlock in completely mundane situations. They also seem dependent on remote operator assistance when encountering non-ideal conditions, greatly limiting their maximum speed. If anything, legislation and social acceptance has moved faster than the technology. That's the opposite what many of us observing this space expected 10 years ago. At this point I'm starting to have doubts about whether the full dream of self-driving cars will even be realized within my lifetime.
- baby 4y ago
  I read this comment after taking a cruise in SF, which is a self driving cab with no driver. It basically reminds me of all the comments saying that VR has no future, written by people who have never tried VR and would get their mind blown by it if they tried the latest iteration. Maybe you should come to SF and try one of these self driving cars yourself :)
  - 10x_contrarian 4y ago
    I actually do live in the Bay Area and spend a lot of time in San Francisco. I applied for the Cruise waitlist well over a year ago but have not been accepted. I've tried to organize with friends who have access but we rarely have a reason to go the Richmond or Golden Gate Park after 10PM. The coverage area is very limited. I'm impressed that they're actually offering driverless rides on SF streets, but my point stands. The cars operate only on the slowest streets at the quietest hours. Any problem they encounter is handled by remote operators. I'm not outright dismissive of self-driving cars. I truly want them to exist. I don't even own a car and dislike being behind the wheel. I just don't buy into infinite hype pushed by a revolving door of charlatans. Also I do have a modern VR headset and celebrate the technology. But, to make a similar comparison, the metaverse "ready player one" vision is not within our lifetimes.
    - 4y ago
watwut 4y ago
I really don't think so. And we are not nearly close to that situation either. We have some obvious crashes in cars that are nowhere near to be probably "safer then human". And then we have super confident claims of safety by manufacturer.
semi-extrinsic 4y ago
A bigger problem is this: Say you need to prove to the public that the autonomous car is significantly safer, and you do an apples-to-apples comparison between a hypothetical Level 4/5 car and well designed new Level 2 electric car like a Volvo C40 or a BMW i4. The modern Level 2 car is already today at below 1 fatality per billion vehicle miles travelled (VMT). The autonomous car then needs to be below 0.1 fatalities per billion VMT. Meaning that if you have 1 million vehicles of your make deployed, they each need to have driven 30-40 000 miles autonomously before you have enough statistics! That means proving the safety of a Level 4/5 autonomous system is extremely expensive and slow, and requires significant public adoption before it's proven to be safe. The consequence is that, assuming proven safety is necessary before public adoption, it becomes impossible to prove safety. Another point is that OTA upgrades for autonomy become entirely pointless, as you'll be polluting your statistics if you change the code more frequently than every ~3 years.
- ryanwaggoner 4y ago
  I think you’d use crashes, not fatalities.
- cipheredStones 4y ago
  > The modern Level 2 car is already today at below 1 fatality per billion vehicle miles travelled (VMT) Does this statistic count fatalities to people outside the car? I ask because (in the US, at least) car safety ratings don't take those into consideration.
  - rootusrootus 4y ago
    > in the US, at least) car safety ratings don't take those into consideration Which ratings? The NHTSA does include non-occupant fatalities in their statistics. E.g. https://www.nhtsa.gov/press-releases/2020-traffic-crash-data-fatalities https://www.nhtsa.gov/press-releases/2020-traffic-crash-data...
    - cipheredStones 4y ago
      NHTSA safety ratings (https://www.nhtsa.gov/ratings https://www.nhtsa.gov/ratings) only count people in the car. It looks like they proposed a change to that this spring (https://www.nhtsa.gov/press-releases/five-star-safety-ratings-program-updates-proposed https://www.nhtsa.gov/press-releases/five-star-safety-rating...), but I don't think they've implemented it yet. From that press release: > For the first time ever, [the proposed rating program] includes technology recommendations not only for drivers and passengers but for road users outside the vehicle, like pedestrians. News story on this: https://www.theverge.com/2022/3/3/22960262/nhtsa-ncap-five-star-safety-rating-adas-proposal https://www.theverge.com/2022/3/3/22960262/nhtsa-ncap-five-s...
rwmj 4y ago
My semi-serious suggestion is self-driving cars should be painted bright orange with big squishy bumpers and a maximum 20 mph speed limit. They would still be perfectly useful as taxis in big cities but it would greatly limit the damage they could do to anyone.
- rootusrootus 4y ago
  You joke, but this isn't a terrible suggestion. Mercedes has the best answer, in my opinion. Speed limited, carefully controlled environments, full liability resting on the manufacturer. I really don't like how Tesla is foisting off all the beta testing on their customers, pushing that risk to them and other people on (or near) the road, and then also giving them 100% of the liability too. The incentives are misaligned. Tesla should want to make the software better not just to attract some more customers, but because if they screw it up they're going to be on the hook.
tim333 4y ago
So far people have been pretty good about accepting self driving cars, even the Teslas on autopilot that crash into parked trucks quite regularly.
- recursive 4y ago
  > quite regularly How many times do you think this has happened?
  - tim333 4y ago
    I don't know - I must have seen five or so headlines in the papers but don't really follow the stuff. Here's one https://www.youtube.com/watch?v=LfmAG4dk-rU https://www.youtube.com/watch?v=LfmAG4dk-rU
amelius 4y ago
> will have 1/10th of the fatal accidents I think I'm relatively safe from cars on the sidewalk. Yet with fsd cars I'm not so sure anymore.
- recursive 4y ago
  > I think I'm relatively safe from cars on the sidewalk. Not even true. I've been hit by a car (with human driver) on a sidewalk. Driveways cross sidewalks, and drivers seem particularly inattentive when crossing them. When you're walking on the sidewalk, do you ever walk somewhere that's more than one block away? If so, you're going to have to cross a street anyway.
amelius 4y ago
It's not necessarily true for people who think they are in the 99th percentile of best drivers.
qwezxcrty 4y ago
Why self-driving trains are much easier to implement, yet there are not that many systems capable of doing that? Many newer metro lines are GoA 2 or 3, theoretically capable of running autonomously, but they always require a driver in the loop. My partial answer is, making an extremely reliable system is hard. If someone wrote a deadly bug even only happen at a very corner case, it still can kill people. And it's quite hard to prove there isn't such bugs.
- konschubert 4y ago
  Regulations. Nobody ever got fired for buying microsoft or for requiring a driver in the seat.
- gretch 4y ago
  We don’t half self driving trains because there’s far, far less incentive. You only need 1 or 2 drivers for a huge train carrying a lot of ppl/stuff. Optimizing away the driving barely reduces the cost of operating the train as a whole. With cars, everyone drives themselves. There’s a lot more driving happening and so if you can automate that away, you create more value.
brightball 4y ago
Another part of this is that driving is a lot of fun. People (like me) really enjoy it and definitely wouldn’t give it up easily.
- wefarrell 4y ago
  Piggybacking on that sentiment, car ownership becomes less an expression of individuality when cars are driving themselves. No point in owning an expensive sports car when it's the one doing the driving. Ride-sharing and fractional ownership start to make more sense than owning the car outright.
cbsmith 4y ago
This is the gist of it, but you have to ask yourself why it's just the accepted wisdom that it's okay to have the massive level of failure we have now.
paxys 4y ago
Yup, exactly. "There was a 1% national accident rate before autopilot but now it is 0.5%, aren't things great?" Not really, because my personal accident risk just went up from 0.1% to 0.5%.
- Aeolun 4y ago
  But you have no issues with paying for social insurance when you never use it yourself? Isn't this the same kind of tradeoff?
  - paxys 4y ago
    How is it the same tradeoff? I'm not increasing my chances of getting sick by buying insurance. The worst case is that I won't use it, which is fine. The worst case for autopilot is that I'll get into a crash because the camera couldn't see a massive truck in front of me because of glare and shoddy software.
j2kun 4y ago
> I think the problem we have with self-driving cars is more social than technological at this point. > it's a hypothetical so give me some leeway on this! IMO you should not base (and broadcast) your opinions about safety on hypothetical statistics. I don't even believe it's true that overall statistics show self-driving is safer than humans. IIRC prior reports showed that companies were selectively picking statistics about safety.
deleted 4y ago
[deleted]
njarboe 4y ago
This is one reason for going to vision only automatic driving systems like Tesla is doing. The system is more likely to fail when a human would also of had a difficult time. Heavy snow, sun blind out, etc. Strange failures due to radar, lidar, and other sensors with not be understood or accepted.
- AlotOfReading 4y ago
  That would be a reasonable argument only if Tesla's image processing were as good as a human brain (it's not) and if their cameras were reasonably comparable to human eyes (they're not). To take the argument to an absurd extreme, you can't cover a car in 240p webcams from 2003 and expect good driving performance. Moreover, other players also have cars covered in cameras. If anyone thought vision only was the best path forward because they couldn't figure out sensor fusion, they'd already have done it and saved the BOM cost. For what it's worth, my experience has been that cameras are one of the more problematic sensor systems overall. Vendor software is garbage, any particular tuning is finicky in extreme conditions, you have to clean the damn things, cameras streams take up lots of bandwidth, etc.
cryptoegorophy 4y ago
Can this be compared to seat belts? If seat belts needed to prevent injury in every single situation then we would claim that they are ineffective because there is always a chance that seat belt may prevent someone from flying out of the window and not burning in the car being strapped unconscious. Is the goal here to be 100% reliable with 0 deaths or achieve better stats than 46k deaths per year in USA? This now becomes more of a philosophical question.
SamuelAdams 4y ago
> will have 1/10th of the fatal accidents that human drivers would have[0]. If this becomes true then society would have an even bigger problem with organ donations - available supply would plummet. Some of the largest sources of organ donations are from car crash victims.
- buzzerbetrayed 4y ago
  I always hear people cite this statistic and I’m never sure what they’re getting at with it. Surely we’re in agreement that this is a good thing, right? That a shortage of organ donors due to the donors not dying is a good thing. Sometimes it feels like people are suggesting otherwise and I can’t fathom the logic.
  - MertsA 4y ago
    I think there's a slightly charitable reading of that opinion. Replacement organs are already in short supply, it's a real problem. By "bigger problem" he's referring to bigger than it already is, not that organ recipients dying is a worse problem than all of the organ donors dying like the status quo. It's a good problem to have, but it's still something the medical profession is going to have to grapple with when their supply dries up.
pauljurczak 4y ago
It is not only a social, but a legal issue, too. If a human kills someone in an easily avoidable accident, breaking the traffic laws, he may go to jail and/or lose his driving privileges for a while. If an artificial neural network does the same, should it loose driving privileges for a while? Should someone go to jail?
eru 4y ago
I agree. > But, the ones they do have are accidents a human driver would almost certainly have avoided. I suspect most human-driver accidents are also accidents that (other) human drivers almost certainly would have avoided. That's scant consolation for all the people dying in traffic accidents each day, of course.
cmurf 4y ago
I think this is more true than not. But also underestimates the technical problem. Airplanes are not fully autonomous, even in the instrument flight rules system which is highly standardized. You don't have non-standard or non-predictable things. In instrument conditions, only instrument planes and pilots exist. While portions of segments are automated, transitions between segments and phases of flight aren't. It's ripe for automation, yet isn't fully automated. The automobile environment has more objects thus more density of complexity, more non-standard and non-predictable actors like non-autonomous vehicles along with bicycles, mopeds, pedestrians, etc.
patrick451 4y ago
> Until Waymo's cars are better than most humans in every single situation, they won't be able to win over the public perception war. The current situation as basically the opposite: waymo's cars are better than humans in almost zero situations. It's hard to gain the my trust when your car can barely drive in a drizzle.
jjav 4y ago
> But, the ones they do have are accidents a human driver would almost certainly have avoided. > Is this something we would accept? No, and that's good. Human drivers behave mostly like humans, even the worst ones. We all have millenia of evolution fine-tuned to recognize human behavior based on the most subtle of cues. So human bad driving is recognizable and thus far more avoidable. AI drivers are effectively an alies species who make errors that make no sense whatsoever to a human mind, thus they behave, for all practical purposes, apparently completely randomly.

by mabbo 4y ago

just-ok 4y ago

I agree with you entirely. Continuing with your hypothetical, even though we’d be 90% safer as a collective, the safety of the individual feels compromised: the risk of an accident is non-uniform when involving humans (depending on e.g. age, experience, safety, alertness, etc.), but becomes uniform (or at least more uniform) with an algorithm in charge. That’s a tough thing for people to buy into.

_cs2017_ 4y ago
A very good observation. Based on your comment, I think we can relax the requirement stated by OP by saying: "Until Waymo's cars reduce any individual's chance of an accident." So for example, suppose a Waymo car is better than humans overall, but tends to do worse than humans when there's a small bump on the road. And suppose that all humans (in a given regulator's area, e.g., California) tend to encounter such bumps at roughly the same rate (per mile driven) over their lifetime. In that case, it's probably going to be acceptable, since every individual is better off. I don't know, maybe this is not impactful / obvious enough for people to care about? What certainly is obvious is that the safest drivers are much safer than an average driver (does anyone know of a study that estimates this ratio?). Therefore, at the very least, the threshold for Waymo should be not the average accident rate, but the accident rate for the safest drivers.
jsbg 4y ago
> the safety of the individual feels compromised Exactly. I have had zero accidents in 20 years; I'm not interested in a car that will lower the overall accident rate if it increases mine.
- mertd 4y ago
  Even the most perfect defensive driving won't fully mitigate the unsafe driving of others though. I'd rather have them to be in autonomous vehicles.
  - xboxnolifes 4y ago
    Sure, but nobody thinks they are the bad drivers.
  - jjav 4y ago
    > I'd rather have them to be in autonomous vehicles. Who is "them" and who allocates the groups? We can't even get any kind of car control skill testing or training to be part of a driver license issuance.

alphabetting 4y ago

If 10% of the time they got into accidents humans would have avoided we wouldn't be where we are today. I can't imagine any scenarios where these cars get in accidents that a human would have certainly avoided. You also say avoided accidents don't make the news but I'm pretty sure footage of them avoiding accidents that humans would have no chance of will be a major part of their marketing.

RhodesianHunter 4y ago
>I can't imagine any scenarios where these cars get in accidents that a human would have certainly avoided. Then you've not been following the space. The one that immediately comes into mind is the Tesla that slammed into the side of the semi truck because it was painted blue like the sky.
- alphabetting 4y ago
  I've been following it pretty closely. I don't consider Teslas to be anywhere near self driving. By saying "these cars" I was referring to Waymo.
jjav 4y ago
> I can't imagine any scenarios where these cars get in accidents that a human would have certainly avoided. If you follow the events with self-driving accidents, most of them are nonsensical crashes that no human would ever have done.
- alphabetting 4y ago
  I follow AV news very closely. I'm talking about Waymo. The small set of accidents they've been accidents in have been limited to fault of other drivers with the vast majority being rear endings

alexose 4y ago

This is one reason I never understood Tesla's vision-based approach. In order to be accepted, self-driving cars don't need to be just somewhat better than humans most of the time. They need to vastly better in every situation, as you mention, to the point that they'll need every sensory advantage they can get. I got out of a ticket once because I didn't see a "no through traffic" sign against a bright sunset. No chance that same cop gives a self-driving car a pass, nor should he.

szundi 4y ago
Karpathy claimed they worked for years and could not reap the benefits from multiple sensors however hard they tried. He seemed really convinced and does not get to me as one who tells stuff just to justify cost reductions, like Elon sometimes is carried away.
- muglug 4y ago
  It’s also possible they weren’t able to see benefits given the processors that were available to them at the time. It’s also possible that they were so far behind Waymo in the journey to FSD that they weren’t yet at a point where multiple sensors would make a significant difference.
  - alexose 4y ago
    This is my read on it as well. I listened to the same interview (it was definitely on the front page, if not at #1 for a while). I stepped away feeling like Karpathy had described a lot of good business reasons for not using other sensors, but not a lot of good technical reasons. Sensor fusion is hard, yes, but maybe not harder than perfectly re-projecting 2D pixel images into 3D vector space. Just my interpretation, but it felt to me like a hail mary because they were fully committed to being the first mover. Waiting around for LiDAR prices to come down would have meant that Waymo would have beat them.
- impulser_ 4y ago
  I always thought it came down to two reason, costs and looks. Telsa has to sell a car people want to drive daily. They can't have a bunch of lidar sensors on their cars no one would buy them even if they could drive themselves. Also they would most likely cost a lot more due to lidar sensors not being cheap compared to normal cameras. Unlike Waymo who doesn't care about selling cars to people to drive daily. No one going to care that the taxi they are taking looks ugly as long as it get them to the place they are going for cheaper. The cost is also a less of a factor due to them being able to produce an income by charging people to ride in them.

gitfan86 4y ago

It is a valid point but the financial incentives are so big that some jurisdictions will allow it. In fact they already do allow these autonomous systems on public roads. That is going to continue to expand and since the financial incentives are huge even when deaths happen the governments will continue to allow it. And in fact some regulators fully understand the tradeoffs and will prefer autonomy for the better good of the public. An example of this is the Boeing 737 Max, those crashes wouldn't have happened if there were no autopilot systems. But regulators are not suggesting that all autonomous systems on planes be turned off because of the safety and financial advantages of keeping them in place even though they are obviously not perfect.

pauljurczak 4y ago
> Boeing 737 Max, those crashes wouldn't have happened if there were no autopilot systems. Bad example. MCAS was an obvious case of a criminal corporate behavior, not a tradeoff between overall safety vs. technical perfection.
- gitfan86 4y ago
  It doesn't matter, the point is that relying on autopilot will open up crash risks that do not exist with manual flight controls
  - pauljurczak 4y ago
    It does matter! I'm assuming that any sane person, especially a highly trained professional, is not fully relying on automation and has at least some idea of its limitations. In the case of MCAS, corporate fraud distorted the level of trust pilots had in automation, leading to loss of life.

yyyk 4y ago

I agree it's a social problem, but IMHO it's a rather different social problem: Current cars, roads and car culture is adapted to human drivers, and AI is expected to be able to integrate into that. What if we would make cars and rules that are adapted to AI cars and ignore human drivers? e.g. Ban human drivers from some roads, allow AI cars with designs that exploit AI advantages (e.g. much better reaction time) but do not require or even allow human backup (enabling us to put the passengers in a secured shell), etc. I suspect we could than reach a 1/20th rate today.

lynx23 4y ago

You're hitting the nail spot on with this one for me. As a blind pedestrian, I very much feel I am in danger of falling into exactly that group of potential victims you are hinting at. Right now, I have the illusive comfort of the "Vertrauensgrundsaatz" which basically tells every driver obtaining a drivers licence that they need to take special care when it comes to disabled pedestrians. Sure, one might say these new self driving systems will "just" have to follow that same rule as well, but I am very much doubtful this is technically possible. So currently, I feel like the drive to put innovation on the streets and pull money out of pockets is actually actively endangering me in the future. Not very bright outlook I must say.

nicbou 4y ago
By your use of German, I assume that you experience a far higher driving standard than people in North America and frankly most of the world, and even that is far from perfect. The Vertrauensgrundsatz does not account for distracted, tired and inebriated drivers.
- deleted 4y ago
  [deleted]
jjeaff 4y ago
My guess is that self driving cars are already far, far better than human drivers at not hitting pedestrians.
- krferriter 4y ago
  My guess is that it depends a lot on the driver. There are a lot of really reckless drivers, or really bad drivers out there on the roads right now. I know people who've been in several (relatively minor) car crashes, driving the same-ish sort of route to/from college and work that I was during the same period. I think they're just a bad driver. I feel like they're driving recklessly when I'm a passenger in a car they're driving. So a self-driving car might be a better driver than them, while also being worse than the median driver, and much much worse than like the upper portion of the human driver spectrum, like the 1st quintile of the 20% of human drivers who are pretty good. Given how reckless some drivers are, I'd imagine the distribution of crashes among drivers is very skewed. I don't know numbers, but I'd expect to see something like 80% of car crashes involving (or objectively caused by) 20% of the drivers, with most car crashes involving people who have been involved in numerous crashes, and a large minority of drivers having been involved in zero car crashes.

GaryNumanVevo 4y ago

To me, self-driving seems like a band-aid fix for terrible car infrastructure. Driving a car is already one of the most dangerous things the average American can do. Due in part to larger, heavier vehicles, high speeds in residential areas, etc. Even if you had a "perfect" driver that doesn't prevent someone from ramming into you.

rootusrootus 4y ago
> one of the most dangerous things the average American can do About comparable to the risk of falling. Lower risk than suicide. Or death by opioid overdose. And of course, the most dangerous thing most Americans do, by a huge huge huge margin, is overeat and lounge on the couch.
Supercoolfaggot 4y ago
Imagine thinking a picture of a penis is sexual harassment...

larusso 4y ago

As a software engineer myself they will always lose the argument. I saw so many smart systems falter in some weird way that I would never trust a software system completely. I drive an EV with some AI based system that automatically throttles the car etc. But to trust my life and my Familie in the hands of this system (or any other) no thank you.

taeric 4y ago

To double down further on the social side of this, public transit is largely getting there faster than point to point driving. In that many trains and such are already largely "hands off the wheels" for operation. Relatedly, another "problem" with "self-driving" cars is that we want all of the convenience and ease of use, without adjusting liability and ownership considerations. Consider, if Waymo gets to the point where they have a self driving car that you have to have a subscription to use, do you own the car? Are you the liable for any accidents it has? To lean in on that hypothetical. I'd imagine a lot of families will use self driving cars to send kids to school. Is effectively a bus that terminates at your house. Who is liable for a mistake if the operation of it is completely remote?

amf12 4y ago
> Consider, if Waymo gets to the point where they have a self driving car that you have to have a subscription to use, do you own the car? Are you the liable for any accidents it has? I'd imagine the company would take on liability, as long as humans can't drive the vehicle or they aren't driving when the accident occurred. Mercedes already got the ball rolling on this [1]. [1] https://www.roadandtrack.com/news/a39481699/what-happens-if-mercedes-drivepilot-causes-a-crash/ https://www.roadandtrack.com/news/a39481699/what-happens-if-...
- taeric 4y ago
  I can't see that path getting taken without you basically losing ownership of the car?
  - amf12 4y ago
    Agreed, but I can see it being solely a "lease" model. For companies to take on liability, they'd have to make sure a customer doesn't modify the vehicle and keeps up with the maintenance. They might also want the authority to disable "self driving" on non-compliant vehicles.
rootusrootus 4y ago
> public transit is largely getting there faster than point to point driving Aside from some niche cases in very dense cities, is this generally true anywhere? I've visited a lot of cities with various levels of public transit, and I can't think of many where it was faster. More convenient sometimes, sure, cheaper, yep, but faster? Not often.

10x_contrarian 4y ago

I think you're mistaken. There are still massive technological challenges. I have seen nearly no evidence that current self-driving car technology is even remotely close to matching the ability of a novice human driver. Sure while the "don't crash into things" algorithms may generally be fine, these systems seem to frequently deadlock in completely mundane situations. They also seem dependent on remote operator assistance when encountering non-ideal conditions, greatly limiting their maximum speed. If anything, legislation and social acceptance has moved faster than the technology. That's the opposite what many of us observing this space expected 10 years ago. At this point I'm starting to have doubts about whether the full dream of self-driving cars will even be realized within my lifetime.

baby 4y ago
I read this comment after taking a cruise in SF, which is a self driving cab with no driver. It basically reminds me of all the comments saying that VR has no future, written by people who have never tried VR and would get their mind blown by it if they tried the latest iteration. Maybe you should come to SF and try one of these self driving cars yourself :)
- 10x_contrarian 4y ago
  I actually do live in the Bay Area and spend a lot of time in San Francisco. I applied for the Cruise waitlist well over a year ago but have not been accepted. I've tried to organize with friends who have access but we rarely have a reason to go the Richmond or Golden Gate Park after 10PM. The coverage area is very limited. I'm impressed that they're actually offering driverless rides on SF streets, but my point stands. The cars operate only on the slowest streets at the quietest hours. Any problem they encounter is handled by remote operators. I'm not outright dismissive of self-driving cars. I truly want them to exist. I don't even own a car and dislike being behind the wheel. I just don't buy into infinite hype pushed by a revolving door of charlatans. Also I do have a modern VR headset and celebrate the technology. But, to make a similar comparison, the metaverse "ready player one" vision is not within our lifetimes.
  - 4y ago

watwut 4y ago

I really don't think so. And we are not nearly close to that situation either. We have some obvious crashes in cars that are nowhere near to be probably "safer then human". And then we have super confident claims of safety by manufacturer.

semi-extrinsic 4y ago

A bigger problem is this: Say you need to prove to the public that the autonomous car is significantly safer, and you do an apples-to-apples comparison between a hypothetical Level 4/5 car and well designed new Level 2 electric car like a Volvo C40 or a BMW i4. The modern Level 2 car is already today at below 1 fatality per billion vehicle miles travelled (VMT). The autonomous car then needs to be below 0.1 fatalities per billion VMT. Meaning that if you have 1 million vehicles of your make deployed, they each need to have driven 30-40 000 miles autonomously before you have enough statistics! That means proving the safety of a Level 4/5 autonomous system is extremely expensive and slow, and requires significant public adoption before it's proven to be safe. The consequence is that, assuming proven safety is necessary before public adoption, it becomes impossible to prove safety. Another point is that OTA upgrades for autonomy become entirely pointless, as you'll be polluting your statistics if you change the code more frequently than every ~3 years.

ryanwaggoner 4y ago
I think you’d use crashes, not fatalities.
cipheredStones 4y ago
> The modern Level 2 car is already today at below 1 fatality per billion vehicle miles travelled (VMT) Does this statistic count fatalities to people outside the car? I ask because (in the US, at least) car safety ratings don't take those into consideration.
- rootusrootus 4y ago
  > in the US, at least) car safety ratings don't take those into consideration Which ratings? The NHTSA does include non-occupant fatalities in their statistics. E.g. https://www.nhtsa.gov/press-releases/2020-traffic-crash-data-fatalities https://www.nhtsa.gov/press-releases/2020-traffic-crash-data...
  - cipheredStones 4y ago
    NHTSA safety ratings (https://www.nhtsa.gov/ratings https://www.nhtsa.gov/ratings) only count people in the car. It looks like they proposed a change to that this spring (https://www.nhtsa.gov/press-releases/five-star-safety-ratings-program-updates-proposed https://www.nhtsa.gov/press-releases/five-star-safety-rating...), but I don't think they've implemented it yet. From that press release: > For the first time ever, [the proposed rating program] includes technology recommendations not only for drivers and passengers but for road users outside the vehicle, like pedestrians. News story on this: https://www.theverge.com/2022/3/3/22960262/nhtsa-ncap-five-star-safety-rating-adas-proposal https://www.theverge.com/2022/3/3/22960262/nhtsa-ncap-five-s...

rwmj 4y ago

My semi-serious suggestion is self-driving cars should be painted bright orange with big squishy bumpers and a maximum 20 mph speed limit. They would still be perfectly useful as taxis in big cities but it would greatly limit the damage they could do to anyone.

rootusrootus 4y ago
You joke, but this isn't a terrible suggestion. Mercedes has the best answer, in my opinion. Speed limited, carefully controlled environments, full liability resting on the manufacturer. I really don't like how Tesla is foisting off all the beta testing on their customers, pushing that risk to them and other people on (or near) the road, and then also giving them 100% of the liability too. The incentives are misaligned. Tesla should want to make the software better not just to attract some more customers, but because if they screw it up they're going to be on the hook.

tim333 4y ago

So far people have been pretty good about accepting self driving cars, even the Teslas on autopilot that crash into parked trucks quite regularly.

recursive 4y ago
> quite regularly How many times do you think this has happened?
- tim333 4y ago
  I don't know - I must have seen five or so headlines in the papers but don't really follow the stuff. Here's one https://www.youtube.com/watch?v=LfmAG4dk-rU https://www.youtube.com/watch?v=LfmAG4dk-rU

amelius 4y ago

> will have 1/10th of the fatal accidents I think I'm relatively safe from cars on the sidewalk. Yet with fsd cars I'm not so sure anymore.

recursive 4y ago
> I think I'm relatively safe from cars on the sidewalk. Not even true. I've been hit by a car (with human driver) on a sidewalk. Driveways cross sidewalks, and drivers seem particularly inattentive when crossing them. When you're walking on the sidewalk, do you ever walk somewhere that's more than one block away? If so, you're going to have to cross a street anyway.

amelius 4y ago

It's not necessarily true for people who think they are in the 99th percentile of best drivers.

qwezxcrty 4y ago

Why self-driving trains are much easier to implement, yet there are not that many systems capable of doing that? Many newer metro lines are GoA 2 or 3, theoretically capable of running autonomously, but they always require a driver in the loop. My partial answer is, making an extremely reliable system is hard. If someone wrote a deadly bug even only happen at a very corner case, it still can kill people. And it's quite hard to prove there isn't such bugs.

konschubert 4y ago
Regulations. Nobody ever got fired for buying microsoft or for requiring a driver in the seat.
gretch 4y ago
We don’t half self driving trains because there’s far, far less incentive. You only need 1 or 2 drivers for a huge train carrying a lot of ppl/stuff. Optimizing away the driving barely reduces the cost of operating the train as a whole. With cars, everyone drives themselves. There’s a lot more driving happening and so if you can automate that away, you create more value.

brightball 4y ago

Another part of this is that driving is a lot of fun. People (like me) really enjoy it and definitely wouldn’t give it up easily.

wefarrell 4y ago
Piggybacking on that sentiment, car ownership becomes less an expression of individuality when cars are driving themselves. No point in owning an expensive sports car when it's the one doing the driving. Ride-sharing and fractional ownership start to make more sense than owning the car outright.

cbsmith 4y ago

This is the gist of it, but you have to ask yourself why it's just the accepted wisdom that it's okay to have the massive level of failure we have now.

paxys 4y ago

Yup, exactly. "There was a 1% national accident rate before autopilot but now it is 0.5%, aren't things great?" Not really, because my personal accident risk just went up from 0.1% to 0.5%.

Aeolun 4y ago
But you have no issues with paying for social insurance when you never use it yourself? Isn't this the same kind of tradeoff?
- paxys 4y ago
  How is it the same tradeoff? I'm not increasing my chances of getting sick by buying insurance. The worst case is that I won't use it, which is fine. The worst case for autopilot is that I'll get into a crash because the camera couldn't see a massive truck in front of me because of glare and shoddy software.

j2kun 4y ago

> I think the problem we have with self-driving cars is more social than technological at this point. > it's a hypothetical so give me some leeway on this! IMO you should not base (and broadcast) your opinions about safety on hypothetical statistics. I don't even believe it's true that overall statistics show self-driving is safer than humans. IIRC prior reports showed that companies were selectively picking statistics about safety.

deleted 4y ago

[deleted]

njarboe 4y ago

This is one reason for going to vision only automatic driving systems like Tesla is doing. The system is more likely to fail when a human would also of had a difficult time. Heavy snow, sun blind out, etc. Strange failures due to radar, lidar, and other sensors with not be understood or accepted.

AlotOfReading 4y ago
That would be a reasonable argument only if Tesla's image processing were as good as a human brain (it's not) and if their cameras were reasonably comparable to human eyes (they're not). To take the argument to an absurd extreme, you can't cover a car in 240p webcams from 2003 and expect good driving performance. Moreover, other players also have cars covered in cameras. If anyone thought vision only was the best path forward because they couldn't figure out sensor fusion, they'd already have done it and saved the BOM cost. For what it's worth, my experience has been that cameras are one of the more problematic sensor systems overall. Vendor software is garbage, any particular tuning is finicky in extreme conditions, you have to clean the damn things, cameras streams take up lots of bandwidth, etc.

cryptoegorophy 4y ago

Can this be compared to seat belts? If seat belts needed to prevent injury in every single situation then we would claim that they are ineffective because there is always a chance that seat belt may prevent someone from flying out of the window and not burning in the car being strapped unconscious. Is the goal here to be 100% reliable with 0 deaths or achieve better stats than 46k deaths per year in USA? This now becomes more of a philosophical question.

SamuelAdams 4y ago

> will have 1/10th of the fatal accidents that human drivers would have[0]. If this becomes true then society would have an even bigger problem with organ donations - available supply would plummet. Some of the largest sources of organ donations are from car crash victims.

buzzerbetrayed 4y ago
I always hear people cite this statistic and I’m never sure what they’re getting at with it. Surely we’re in agreement that this is a good thing, right? That a shortage of organ donors due to the donors not dying is a good thing. Sometimes it feels like people are suggesting otherwise and I can’t fathom the logic.
- MertsA 4y ago
  I think there's a slightly charitable reading of that opinion. Replacement organs are already in short supply, it's a real problem. By "bigger problem" he's referring to bigger than it already is, not that organ recipients dying is a worse problem than all of the organ donors dying like the status quo. It's a good problem to have, but it's still something the medical profession is going to have to grapple with when their supply dries up.

pauljurczak 4y ago

It is not only a social, but a legal issue, too. If a human kills someone in an easily avoidable accident, breaking the traffic laws, he may go to jail and/or lose his driving privileges for a while. If an artificial neural network does the same, should it loose driving privileges for a while? Should someone go to jail?

eru 4y ago

I agree. > But, the ones they do have are accidents a human driver would almost certainly have avoided. I suspect most human-driver accidents are also accidents that (other) human drivers almost certainly would have avoided. That's scant consolation for all the people dying in traffic accidents each day, of course.

cmurf 4y ago

I think this is more true than not. But also underestimates the technical problem. Airplanes are not fully autonomous, even in the instrument flight rules system which is highly standardized. You don't have non-standard or non-predictable things. In instrument conditions, only instrument planes and pilots exist. While portions of segments are automated, transitions between segments and phases of flight aren't. It's ripe for automation, yet isn't fully automated. The automobile environment has more objects thus more density of complexity, more non-standard and non-predictable actors like non-autonomous vehicles along with bicycles, mopeds, pedestrians, etc.

patrick451 4y ago

> Until Waymo's cars are better than most humans in every single situation, they won't be able to win over the public perception war. The current situation as basically the opposite: waymo's cars are better than humans in almost zero situations. It's hard to gain the my trust when your car can barely drive in a drizzle.

jjav 4y ago

> But, the ones they do have are accidents a human driver would almost certainly have avoided. > Is this something we would accept? No, and that's good. Human drivers behave mostly like humans, even the worst ones. We all have millenia of evolution fine-tuned to recognize human behavior based on the most subtle of cues. So human bad driving is recognizable and thus far more avoidable. AI drivers are effectively an alies species who make errors that make no sense whatsoever to a human mind, thus they behave, for all practical purposes, apparently completely randomly.