5 ms·

The desire to have a car that drives itself is so high that it overrides folks' self preservation. They'll pay kilodollars for unsafe systems. Regulation is the

The desire to have a car that drives itself is so high that it overrides folks' self preservation. They'll pay kilodollars for unsafe systems. Regulation is the only solution I can see to prevent buggy beta software unleashing mayhem on the road.

deadbabe 2y ago
It’s unfortunate that they are not only putting themselves in danger, but also the lives of people all around them.
- shadowgovt 2y ago
  But that's every time a human gets behind the wheel also. It comes down to trust: do you trust a machine to get it right every time, or a human to get it right every time? I think people will answer differently based on their differing experiences with machines and people. (... but to my money: when the machine malfunctions, we can crack open its black box, look at its analysis logs, find errors, and update every other clone of the machine to never make that mistake again. Can't do that with humans; for all the lessons we learn about road safety, we are trying inefficiently to imprint them on a new, naive crop of drivers every day).
  - deadbabe 2y ago
    It’s not that simple! Machines distribute accidents without bias. Individuals who are otherwise safe drivers could crash viciously if using self driving modes. Sometimes you can develop intuition for what cars are being driven by bad human drivers and you can avoid them. Machines will appear to be driving just fine until suddenly they’re not, and this can happen very quickly. Also, if you know a car is being driven by a human rather than a machine, you can use your empathy to anticipate what a human driver is going to do in a situation and prepare for that, where as a machine can behave suddenly and unpredictably, increasing your chances of engaging in an accident. Keep these self driving vehicles in their own lanes.
    - shadowgovt 2y ago
      > Machines distribute accidents without bias Isn't that a good thing? Or are we assuming we have control over the other drivers who are bad drivers and will kill us when they wreck into us? > Also, if you know a car is being driven by a human rather than a machine, you can use your empathy to anticipate what a human driver is going to do in a situation and prepare for that People keep claiming this but I think it's a claim without sufficient evidence. I've watched humans go backwards up a one-way street. I've watched humans stop at three stop signs and miss the forth. I've watched humans drive perfectly up until the point they had a coronary. I suspect it's something people like to believe because they get to say "I avoided that idiot swerving on the highway; that kept me safe" and they miss the huge confirmation bias that they're really safe because they didn't get wiped out by the guy who had the heart attack and suddenly drifted into the oncoming traffic lane, leaving no time to react. It's an illusion-of-safety feeling that, ironically, comes from so many human drivers being erratic in non-immediately-dangerous ways, far more often than an automated vehicle is erratic at all. We don't hear a lot about from the tens of thousands of drivers killed on the road every year failed to use empathy to avoid that crash, because they're dead. So we're missing a huge datapoint to evaluate the "humans can communicate with each other empathetically on the road so they're safer than sharing the road with automata" claim.
  - rented_mule 2y ago
    For me, it's more nuanced than this. Most mistakes that are made by human drivers around me make intuitive sense to me based on how they are driving. They are often somewhat predictable. In those cases, I can partially mitigate those ahead of time (not always - I've been broadsided where I didn't see the other car until the last moment). The wide adoption of cell phones required a significant update to my mental model, but you can spot things like lane drift, uneven speed, etc. that tell you to give space to someone because they are probably distracted. Both chemically-impaired and sleep-deprived drivers tend to give even more clues to stay clear. I find predictions informed by observation much more difficult with automated driving systems. In many cases, such cars will appear to be driving perfectly, right up until a severe mistake is made. Instead of having observable clues, it requires an understanding of what sensor suite and what version of software is in which vehicle in order to understand the tendencies of that vehicle. Something we can't possibly keep up with, especially given that the developers of the machine learning models being deployed can't deeply characterize each deployment's failure modes. Given all that, I feel more able to defend myself against human mistakes than automated mistakes.
eagerpace 2y ago
Have you used it? I find that when you use it as designed and pay attention it is much safer and easier to drive. Especially in road trips I arrive fresher than I would if I were manually driving the entire time. Certainly there are dangers but there are already huge benefits.
- bryanlarsen 2y ago
  For me the big advantage of Tesla FSD is that I find I spend more attention watching for moose in rural areas and pedestrians/situational awareness in urban areas because I can let the computer handle the mundane stuff. Is this also the case with Blue Cruise? Does the eyeball monitoring camera count watching ditches & sidewalks as "paying attention" or does it force you to strictly keep eyes front?
  - drcode 2y ago
    yeah the combination of fsd 12.3.5 + attentive human is incredible
- davely 2y ago
  Awhile back (ok, 10 years ago!), there was an interesting New Yorker article [1] about the rise of automation, specifically related to flight, but I think some points are relevant here: > But, as pilots were being freed of these responsibilities, they were becoming increasingly susceptible to boredom and complacency--problems that were all the more insidious for being difficult to identify and assess. As one pilot whom Wiener interviewed put it: "I know I'm not in the loop, but I'm not exactly out of the loop. It's more like I'm flying alongside the loop." I think these assisted driving features in cars are the same way. I personally like the idea of them, humans seem to be incredibly horrible drivers (especially here in the Bay Area). However, when attention and monitoring is still required, people get complacent. All that said, I'm not sure what the answer is -- maybe these systems need to get to a point where they are fully autonomous? But how do we get there and feel comfortable about it in the first place? Take air travel for example -- planes are probably perfectly capable of taking off, flying themselves from point A to point B, and landing without human intervention. Would you travel on a plane with no pilot? Probably not! [1] https://www.newyorker.com/science/maria-konnikova/hazards-automation https://www.newyorker.com/science/maria-konnikova/hazards-au...
fallingknife 2y ago
The current mayhem on the road is killing 40K people every year in the US, and I'm not scared of that. So you'll have to excuse me for not caring about this hypothetical software induced mayhem.
- Filligree 2y ago
  In a way, it's useful that American roads are so unsafe. It means self-driving vehicles can be deployed early, in an environment where they can do some good, well prior to when they'd be a net safety improvement in e.g. Norway.
  - mannykannot 2y ago
    On the other hand, the behavior that is largely responsible for the difference may also make the technical challenge more difficult.
ActionHank 2y ago
kilodollars - the metric unit that might just swing the US away from imperial.
- ilinx 2y ago
  Honestly we’ll probably just figure out how much a football field costs and use that.
  - ActionHank 2y ago
    Field-dollars™
- saalweachter 2y ago
  As an American, I've been trying to get people to use the Everest, the amount of money it costs to climb Mount Everest, as the unit of measure for midlife crises and upper class indulgences. (One Everest is around $50,000-$100,000.)
- satiric 2y ago
  I know this was just a lighthearted comment, but for folks like me who like to take things too seriously... "grand" has the same value, and has the benefit of already being in common use.
drcode 2y ago
https://digitalassets.tesla.com/tesla-contents/image/upload/f_auto,q_auto/safety-report-2023-desktop.jpg https://digitalassets.tesla.com/tesla-contents/image/upload/...
- delfinom 2y ago
  Those stats have always seemed game. At the basic level "autopilot technology" is a wide grouping of various marketing wank Tesla invented. Is it or is it not FSD? Because for the longest time it was just driver assistance tech that everyone else has in different forms. Shit, my Civic can self drive just fine using radar cruise control (it even handles merging vehicles) and lane keep. The US average passenger vehicle age is also 13 years. This puts _many_ existing vehicles on the road outside the range of modern driver assistance equipment which of course would affect this chart.
  - neogodless 2y ago
    The chart should show like "miles driven with cruise control enabled" and "miles driven with <mild assist> cruise control enabled" (e.g. radar / laser / automatic, lane assist, etc." for all brands of vehicles.
  - stkdump 2y ago
    You can also get independent data. For example what drivers have to pay for (liability) ensurance based on make and model. The rating system in Germany puts a Model 3 at rating 20 (out of 10-24, where 10 is lowest cost for insurance) this is much worse than many older cars without any assited driving. For example my 10 year old Ford was on 17, my current VW (which has a lot of assisted driving features) is at 16. Of course it is not only the car safety, it is also influenced by the average driver behavior. But it seems in general Tesla isn't doing great when you look at such independent data.
vel0city 2y ago
Reading the facts of the accident central to this investigation, I imagine most meat bag driving systems would have also probably hit the car. It's not like the car ran into a stopped police car with flashing lights on.
- toast0 2y ago
  I don't know if we have enough facts in the Texas case to guess if a human in the same situation would have been able to handle the situation better. All the facts I've seen are there was a vehicle stopped in traffic lanes with its lights off. At least one human driver ahead of the accident car that avoided the stopped vehicle, but was close enough to observe the collision. There's lots of scenarios that fit these facts, some of which are hard for humans; all of which are hard for driver assistance software that ignores stationary objects. Even if an active human driver would be likely to still hit the other car, they may have begun braking sooner and reduce the energy involved in the collision. If the accident car was closing following the observer car, both in the same lane as the stopped car, and the observer car changed lanes to avoid the stopped car 'at the last second', yeah, that's hard for humans too when there's limited visibility ahead of the car in front (which is a good reason to avoid following so closely, but things happen). If it was just light traffic at high speeds and a stopped car, I'd expect an active driver to at least reduce speed significantly. Yes, it was at night and the stopped car had no lights; however, the accident car's headlights should illuminate the stopped car's reflectors if nothing else, and given the observer car had passed the accident car recently, there should have been some light cast from its headlights as well.
  - vel0city 2y ago
    Sure sounds like the Ford was following the observer car, probably pretty closely for the given speed. Having reflectors on the stationary car isn't useful when the stationary car is completely obscured by the observer car. If the observer car missed it by only a couple of seconds, there was probably only five seconds or so for the Ford driver to see it, realize what was happening, and properly respond. And at those speeds and that kind of distance, braking probably wasn't an option especially if you've already burned a second and a half just realizing "oh shit that's a stopped car". I haven't heard either way if there was significant braking by the Ford or not, going on an open Texas highway typically means they were probably going >80mph as the posted speed was probably 80mph. I'm not sure exactly which stretch of I-10 they were on, but a lot of it around San Antonio (especially the unlit stretches) are posted 80. Even if they slammed on the brakes and managed to cut half their speed they're still slamming into a stationary, heavy object at 40ish mph. I imagine it pretty much is an instance of following too closely and going too fast for the lighting conditions for the possibility of suddenly encountering a completely stopped object in the road. I agree, its a bad limitation of "self driving" systems, but in the end I imagine the average human would have probably had a similar following distance and also be travelling at a similar speed. The Ford system does track attentiveness and will drop out of bluecruise mode and bring the car to a stop if the driver isn't paying attention. It does a lot of gaze tracking with IR lights and cameras. Its not like on some cars where attaching a little weight to the wheel is enough to fake attention. The driver was paying attention to the road, at least within the last few seconds. EDIT: Sounds like the stretch in question is a 70mph area. Still, I imagine on an open highway and how loads of Texans drive, they were probably going about 80mph. Also, it does appear this section of I-10 does have roadway lighting. FWIW, I've seen lots of cars slam into the back of a stopped car on a highway in bright daylight conditions without the assistance of driving aids. My old office had a pretty good view of a long stretch of a major 70mph highway; I saw a lot of collisions. Meatbag drivers are pretty terrible.
andrewmutz 2y ago
On the other hand, it's hard to build safe self-driving cars without actually deploying them in the real world. If we can tolerate some level of risk, we can get to self-driving cars sooner. Every year we delay having safe self-driving cars, about 50,000 people die on the road who don't need to die.
- HPsquared 2y ago
  It's basically the trolley problem.
nunez 2y ago
I'd rather have slightly unsafe self-driving cars than completely unsafe human-driven cars where the human is distracted by whatever's on their phone, kids screaming, accidents on the road, etc.

deadbabe 2y ago

It’s unfortunate that they are not only putting themselves in danger, but also the lives of people all around them.

shadowgovt 2y ago
But that's every time a human gets behind the wheel also. It comes down to trust: do you trust a machine to get it right every time, or a human to get it right every time? I think people will answer differently based on their differing experiences with machines and people. (... but to my money: when the machine malfunctions, we can crack open its black box, look at its analysis logs, find errors, and update every other clone of the machine to never make that mistake again. Can't do that with humans; for all the lessons we learn about road safety, we are trying inefficiently to imprint them on a new, naive crop of drivers every day).
- deadbabe 2y ago
  It’s not that simple! Machines distribute accidents without bias. Individuals who are otherwise safe drivers could crash viciously if using self driving modes. Sometimes you can develop intuition for what cars are being driven by bad human drivers and you can avoid them. Machines will appear to be driving just fine until suddenly they’re not, and this can happen very quickly. Also, if you know a car is being driven by a human rather than a machine, you can use your empathy to anticipate what a human driver is going to do in a situation and prepare for that, where as a machine can behave suddenly and unpredictably, increasing your chances of engaging in an accident. Keep these self driving vehicles in their own lanes.
  - shadowgovt 2y ago
    > Machines distribute accidents without bias Isn't that a good thing? Or are we assuming we have control over the other drivers who are bad drivers and will kill us when they wreck into us? > Also, if you know a car is being driven by a human rather than a machine, you can use your empathy to anticipate what a human driver is going to do in a situation and prepare for that People keep claiming this but I think it's a claim without sufficient evidence. I've watched humans go backwards up a one-way street. I've watched humans stop at three stop signs and miss the forth. I've watched humans drive perfectly up until the point they had a coronary. I suspect it's something people like to believe because they get to say "I avoided that idiot swerving on the highway; that kept me safe" and they miss the huge confirmation bias that they're really safe because they didn't get wiped out by the guy who had the heart attack and suddenly drifted into the oncoming traffic lane, leaving no time to react. It's an illusion-of-safety feeling that, ironically, comes from so many human drivers being erratic in non-immediately-dangerous ways, far more often than an automated vehicle is erratic at all. We don't hear a lot about from the tens of thousands of drivers killed on the road every year failed to use empathy to avoid that crash, because they're dead. So we're missing a huge datapoint to evaluate the "humans can communicate with each other empathetically on the road so they're safer than sharing the road with automata" claim.
- rented_mule 2y ago
  For me, it's more nuanced than this. Most mistakes that are made by human drivers around me make intuitive sense to me based on how they are driving. They are often somewhat predictable. In those cases, I can partially mitigate those ahead of time (not always - I've been broadsided where I didn't see the other car until the last moment). The wide adoption of cell phones required a significant update to my mental model, but you can spot things like lane drift, uneven speed, etc. that tell you to give space to someone because they are probably distracted. Both chemically-impaired and sleep-deprived drivers tend to give even more clues to stay clear. I find predictions informed by observation much more difficult with automated driving systems. In many cases, such cars will appear to be driving perfectly, right up until a severe mistake is made. Instead of having observable clues, it requires an understanding of what sensor suite and what version of software is in which vehicle in order to understand the tendencies of that vehicle. Something we can't possibly keep up with, especially given that the developers of the machine learning models being deployed can't deeply characterize each deployment's failure modes. Given all that, I feel more able to defend myself against human mistakes than automated mistakes.

eagerpace 2y ago

Have you used it? I find that when you use it as designed and pay attention it is much safer and easier to drive. Especially in road trips I arrive fresher than I would if I were manually driving the entire time. Certainly there are dangers but there are already huge benefits.

bryanlarsen 2y ago
For me the big advantage of Tesla FSD is that I find I spend more attention watching for moose in rural areas and pedestrians/situational awareness in urban areas because I can let the computer handle the mundane stuff. Is this also the case with Blue Cruise? Does the eyeball monitoring camera count watching ditches & sidewalks as "paying attention" or does it force you to strictly keep eyes front?
- drcode 2y ago
  yeah the combination of fsd 12.3.5 + attentive human is incredible
davely 2y ago
Awhile back (ok, 10 years ago!), there was an interesting New Yorker article [1] about the rise of automation, specifically related to flight, but I think some points are relevant here: > But, as pilots were being freed of these responsibilities, they were becoming increasingly susceptible to boredom and complacency--problems that were all the more insidious for being difficult to identify and assess. As one pilot whom Wiener interviewed put it: "I know I'm not in the loop, but I'm not exactly out of the loop. It's more like I'm flying alongside the loop." I think these assisted driving features in cars are the same way. I personally like the idea of them, humans seem to be incredibly horrible drivers (especially here in the Bay Area). However, when attention and monitoring is still required, people get complacent. All that said, I'm not sure what the answer is -- maybe these systems need to get to a point where they are fully autonomous? But how do we get there and feel comfortable about it in the first place? Take air travel for example -- planes are probably perfectly capable of taking off, flying themselves from point A to point B, and landing without human intervention. Would you travel on a plane with no pilot? Probably not! [1] https://www.newyorker.com/science/maria-konnikova/hazards-automation https://www.newyorker.com/science/maria-konnikova/hazards-au...

fallingknife 2y ago

The current mayhem on the road is killing 40K people every year in the US, and I'm not scared of that. So you'll have to excuse me for not caring about this hypothetical software induced mayhem.

Filligree 2y ago
In a way, it's useful that American roads are so unsafe. It means self-driving vehicles can be deployed early, in an environment where they can do some good, well prior to when they'd be a net safety improvement in e.g. Norway.
- mannykannot 2y ago
  On the other hand, the behavior that is largely responsible for the difference may also make the technical challenge more difficult.

ActionHank 2y ago

kilodollars - the metric unit that might just swing the US away from imperial.

ilinx 2y ago
Honestly we’ll probably just figure out how much a football field costs and use that.
- ActionHank 2y ago
  Field-dollars™
saalweachter 2y ago
As an American, I've been trying to get people to use the Everest, the amount of money it costs to climb Mount Everest, as the unit of measure for midlife crises and upper class indulgences. (One Everest is around $50,000-$100,000.)
satiric 2y ago
I know this was just a lighthearted comment, but for folks like me who like to take things too seriously... "grand" has the same value, and has the benefit of already being in common use.

drcode 2y ago

https://digitalassets.tesla.com/tesla-contents/image/upload/f_auto,q_auto/safety-report-2023-desktop.jpg https://digitalassets.tesla.com/tesla-contents/image/upload/...

delfinom 2y ago
Those stats have always seemed game. At the basic level "autopilot technology" is a wide grouping of various marketing wank Tesla invented. Is it or is it not FSD? Because for the longest time it was just driver assistance tech that everyone else has in different forms. Shit, my Civic can self drive just fine using radar cruise control (it even handles merging vehicles) and lane keep. The US average passenger vehicle age is also 13 years. This puts _many_ existing vehicles on the road outside the range of modern driver assistance equipment which of course would affect this chart.
- neogodless 2y ago
  The chart should show like "miles driven with cruise control enabled" and "miles driven with <mild assist> cruise control enabled" (e.g. radar / laser / automatic, lane assist, etc." for all brands of vehicles.
- stkdump 2y ago
  You can also get independent data. For example what drivers have to pay for (liability) ensurance based on make and model. The rating system in Germany puts a Model 3 at rating 20 (out of 10-24, where 10 is lowest cost for insurance) this is much worse than many older cars without any assited driving. For example my 10 year old Ford was on 17, my current VW (which has a lot of assisted driving features) is at 16. Of course it is not only the car safety, it is also influenced by the average driver behavior. But it seems in general Tesla isn't doing great when you look at such independent data.

vel0city 2y ago

Reading the facts of the accident central to this investigation, I imagine most meat bag driving systems would have also probably hit the car. It's not like the car ran into a stopped police car with flashing lights on.

toast0 2y ago
I don't know if we have enough facts in the Texas case to guess if a human in the same situation would have been able to handle the situation better. All the facts I've seen are there was a vehicle stopped in traffic lanes with its lights off. At least one human driver ahead of the accident car that avoided the stopped vehicle, but was close enough to observe the collision. There's lots of scenarios that fit these facts, some of which are hard for humans; all of which are hard for driver assistance software that ignores stationary objects. Even if an active human driver would be likely to still hit the other car, they may have begun braking sooner and reduce the energy involved in the collision. If the accident car was closing following the observer car, both in the same lane as the stopped car, and the observer car changed lanes to avoid the stopped car 'at the last second', yeah, that's hard for humans too when there's limited visibility ahead of the car in front (which is a good reason to avoid following so closely, but things happen). If it was just light traffic at high speeds and a stopped car, I'd expect an active driver to at least reduce speed significantly. Yes, it was at night and the stopped car had no lights; however, the accident car's headlights should illuminate the stopped car's reflectors if nothing else, and given the observer car had passed the accident car recently, there should have been some light cast from its headlights as well.
- vel0city 2y ago
  Sure sounds like the Ford was following the observer car, probably pretty closely for the given speed. Having reflectors on the stationary car isn't useful when the stationary car is completely obscured by the observer car. If the observer car missed it by only a couple of seconds, there was probably only five seconds or so for the Ford driver to see it, realize what was happening, and properly respond. And at those speeds and that kind of distance, braking probably wasn't an option especially if you've already burned a second and a half just realizing "oh shit that's a stopped car". I haven't heard either way if there was significant braking by the Ford or not, going on an open Texas highway typically means they were probably going >80mph as the posted speed was probably 80mph. I'm not sure exactly which stretch of I-10 they were on, but a lot of it around San Antonio (especially the unlit stretches) are posted 80. Even if they slammed on the brakes and managed to cut half their speed they're still slamming into a stationary, heavy object at 40ish mph. I imagine it pretty much is an instance of following too closely and going too fast for the lighting conditions for the possibility of suddenly encountering a completely stopped object in the road. I agree, its a bad limitation of "self driving" systems, but in the end I imagine the average human would have probably had a similar following distance and also be travelling at a similar speed. The Ford system does track attentiveness and will drop out of bluecruise mode and bring the car to a stop if the driver isn't paying attention. It does a lot of gaze tracking with IR lights and cameras. Its not like on some cars where attaching a little weight to the wheel is enough to fake attention. The driver was paying attention to the road, at least within the last few seconds. EDIT: Sounds like the stretch in question is a 70mph area. Still, I imagine on an open highway and how loads of Texans drive, they were probably going about 80mph. Also, it does appear this section of I-10 does have roadway lighting. FWIW, I've seen lots of cars slam into the back of a stopped car on a highway in bright daylight conditions without the assistance of driving aids. My old office had a pretty good view of a long stretch of a major 70mph highway; I saw a lot of collisions. Meatbag drivers are pretty terrible.

andrewmutz 2y ago

On the other hand, it's hard to build safe self-driving cars without actually deploying them in the real world. If we can tolerate some level of risk, we can get to self-driving cars sooner. Every year we delay having safe self-driving cars, about 50,000 people die on the road who don't need to die.

HPsquared 2y ago
It's basically the trolley problem.

nunez 2y ago

I'd rather have slightly unsafe self-driving cars than completely unsafe human-driven cars where the human is distracted by whatever's on their phone, kids screaming, accidents on the road, etc.