4 ms·

I thought the title meant the software crashed, but it was the car that crashed: "They didn’t make it out of California without crashing into easily avoidable r

by gdulli 1y ago

I thought the title meant the software crashed, but it was the car that crashed: "They didn’t make it out of California without crashing into easily avoidable road debris that badly damaged the Tesla Model Y ... In the video, you can see that the driver doesn’t have his hands on the steering wheel. The passenger spots the debris way ahead of time. There was plenty of time to react, but the driver didn’t get his hands on the steering wheel until the last second."

lazide 1y ago
The issue with ‘driver in the loop only for emergencies’ is that it’s harder for someone to be prepared to react only occasionally (but still promptly and correctly) when outlier behavior happens, not easier. For automated tools like CNC machines, there is a reason it’s just ‘emergency stop’ for instance, not ‘tap the correct thing we should do instead’. But doing an e-stop on a road at speed is a very dangerous action as well, which is why that isn’t an option here either.
- UncleMeat 1y ago
  And you don't even know until it is too late even if you are paying attention. Suppose you see the obstruction well in advance. You expect the car to change lanes. You get past the point where you would have changed lanes but there's still time. You aren't yet aware that the car is missing the obstruction. A moment later you are much closer and make the judgement that the car is making a mistake. You grab the wheel but now it's too late. The alternative is taking control of the car all the time. Something nobody is going to practically do.
  - lazide 1y ago
    Yeah, in many ways the only way a human can know for sure that it’s not seeing something is past the point of no return/the absence of action, which is always hard.
- detaro 1y ago
  And why many "real" self-driving systems that share the road with other users are often quite speed-limited, to allow them to come to a full stop relatively safely if needed, and give a human time to take over properly. (E.g. the Mercedes system will stick way below the usual speed limits, which means it's main use currently is handling heavy traffic with stop-and-go, or many of the autonomous "busses" for dense city spaces/campuses/...) Passively monitoring a situation continuously and reacting quickly when needed is something humans are not good at. Even with airline pilots, where we do invest a lot in training and monitoring, it's a well-understood issue that having to take over in a surprising situation often leads to confusion and time needed to re-orient before they can take effective action. Which for planes is often fine, because you have the time buffer needed for that, but not always.
- anonymars 1y ago
  There's sort of an analogous conundrum with Airbus-style automation: the system has various levels of automation and protection (e.g. preventing you from stalling the plane) And then when something goes wrong, you are unceremoniously handed a plane with potentially some or all of those protections no longer active. As an analogy, imagine your FSD car was trying to slow down for something, but along the way there is some issue with a sensor. So it gives up and hands control back to you while it's in the middle of braking, yet now your ABS is no longer active. So now the situation is much more sudden than it would have been (if you had been driving the car you would have been aware of it and slowing down for it youself earlier in the game), you likely weren't paying as much attention in the first place because of the automation, and some of the normal protection isn't working. So it's almost three levels of adding insult to injury. Potentially related discussion: https://news.ycombinator.com/item?id=43970363 https://news.ycombinator.com/item?id=43970363
  - kayodelycaon 1y ago
    It’s important to point out that airline pilots are trained to handle sudden emergencies. This has been incredibly successful at scale. But it came great expense of both money and lost lives. And it still isn’t perfect. The level of training required to oversee full automation is non-trivial if you have to do more than press a stop button.
    - lazide 1y ago
      Notably, it’s likely harder than… just driving yourself.
  - Coffeewine 1y ago
    I’ll defend airbus a little - there are flight laws that more or less provide at any given moment as much automation as is possible given the state of the sensors and computers. So it doesn’t just go ‘oops, a sensor failed, now you have direct control of the plane.’ It does have the same problem - if 99.999% of your flight time is spent in normal law you are not especially ready to operate in one of the alternate laws or god forbid direct law, which is similar to the case of a driver who perhaps accustomed to the system forget how to drive. But I think we have a ways before we get there. If the car could detect issues earlier and more gradually notify the driver that they need to take control, most every driver at present retains the knowledge of how to directly operate a car with non-navigational automation (abs as you mentioned, power stearing, etc)
catapart 1y ago
And if the car was fully self driving and capable of seeing debris that was clearly seen by two humans (whom tesla claims have inferior eyes), it could have taken the opportunity to change lanes and avoid the obstruction. They want to pretend you'll only need to actually intervene in 'edge case' situations, but here we have an example of perfect conditions requiring intervention. Regardless of the buzzwords they can attach to whatever methods they are using, it doesn't feel like it works.
- rsynnott 1y ago
  > whom tesla claims have inferior eyes ... wait, do they actually claim that? I mean that's just nonsense.
xeromal 1y ago
They were aware of the debris but intervening would lessen the impact of the video. I'm glad they didn't intervene. This is a bad look for FSD.

by gdulli 1y ago

lazide 1y ago

The issue with ‘driver in the loop only for emergencies’ is that it’s harder for someone to be prepared to react only occasionally (but still promptly and correctly) when outlier behavior happens, not easier. For automated tools like CNC machines, there is a reason it’s just ‘emergency stop’ for instance, not ‘tap the correct thing we should do instead’. But doing an e-stop on a road at speed is a very dangerous action as well, which is why that isn’t an option here either.

UncleMeat 1y ago
And you don't even know until it is too late even if you are paying attention. Suppose you see the obstruction well in advance. You expect the car to change lanes. You get past the point where you would have changed lanes but there's still time. You aren't yet aware that the car is missing the obstruction. A moment later you are much closer and make the judgement that the car is making a mistake. You grab the wheel but now it's too late. The alternative is taking control of the car all the time. Something nobody is going to practically do.
- lazide 1y ago
  Yeah, in many ways the only way a human can know for sure that it’s not seeing something is past the point of no return/the absence of action, which is always hard.
detaro 1y ago
And why many "real" self-driving systems that share the road with other users are often quite speed-limited, to allow them to come to a full stop relatively safely if needed, and give a human time to take over properly. (E.g. the Mercedes system will stick way below the usual speed limits, which means it's main use currently is handling heavy traffic with stop-and-go, or many of the autonomous "busses" for dense city spaces/campuses/...) Passively monitoring a situation continuously and reacting quickly when needed is something humans are not good at. Even with airline pilots, where we do invest a lot in training and monitoring, it's a well-understood issue that having to take over in a surprising situation often leads to confusion and time needed to re-orient before they can take effective action. Which for planes is often fine, because you have the time buffer needed for that, but not always.
anonymars 1y ago
There's sort of an analogous conundrum with Airbus-style automation: the system has various levels of automation and protection (e.g. preventing you from stalling the plane) And then when something goes wrong, you are unceremoniously handed a plane with potentially some or all of those protections no longer active. As an analogy, imagine your FSD car was trying to slow down for something, but along the way there is some issue with a sensor. So it gives up and hands control back to you while it's in the middle of braking, yet now your ABS is no longer active. So now the situation is much more sudden than it would have been (if you had been driving the car you would have been aware of it and slowing down for it youself earlier in the game), you likely weren't paying as much attention in the first place because of the automation, and some of the normal protection isn't working. So it's almost three levels of adding insult to injury. Potentially related discussion: https://news.ycombinator.com/item?id=43970363 https://news.ycombinator.com/item?id=43970363
- kayodelycaon 1y ago
  It’s important to point out that airline pilots are trained to handle sudden emergencies. This has been incredibly successful at scale. But it came great expense of both money and lost lives. And it still isn’t perfect. The level of training required to oversee full automation is non-trivial if you have to do more than press a stop button.
  - lazide 1y ago
    Notably, it’s likely harder than… just driving yourself.
- Coffeewine 1y ago
  I’ll defend airbus a little - there are flight laws that more or less provide at any given moment as much automation as is possible given the state of the sensors and computers. So it doesn’t just go ‘oops, a sensor failed, now you have direct control of the plane.’ It does have the same problem - if 99.999% of your flight time is spent in normal law you are not especially ready to operate in one of the alternate laws or god forbid direct law, which is similar to the case of a driver who perhaps accustomed to the system forget how to drive. But I think we have a ways before we get there. If the car could detect issues earlier and more gradually notify the driver that they need to take control, most every driver at present retains the knowledge of how to directly operate a car with non-navigational automation (abs as you mentioned, power stearing, etc)

catapart 1y ago

And if the car was fully self driving and capable of seeing debris that was clearly seen by two humans (whom tesla claims have inferior eyes), it could have taken the opportunity to change lanes and avoid the obstruction. They want to pretend you'll only need to actually intervene in 'edge case' situations, but here we have an example of perfect conditions requiring intervention. Regardless of the buzzwords they can attach to whatever methods they are using, it doesn't feel like it works.

rsynnott 1y ago
> whom tesla claims have inferior eyes ... wait, do they actually claim that? I mean that's just nonsense.

xeromal 1y ago

They were aware of the debris but intervening would lessen the impact of the video. I'm glad they didn't intervene. This is a bad look for FSD.