3 ms·

Circumventing the attention monitoring systems isn’t the issue. Those only exist for the purpose of liability shifting, and serve no other purpose. We _know_ f

by olliej 3y ago

Circumventing the attention monitoring systems isn’t the issue. Those only exist for the purpose of liability shifting, and serve no other purpose. We _know_ from innumerable studies, and just direct experience in aviation that saying “this will be done automatically” but also requiring absolute attention is not something humans can do. The only people trying to claim to the contrary are the self driving car people, and only so that when their self driving system fails they can say “the person in the driver seat is the person actually driving and it’s their fault, not our faulty software”. This is without considering their other failure mode: disabling without warning, often immediately prior to crashing. Autopilots in aircraft can do “something’s wrong I’m disengaging and warning the pilots” because any time the autopilot is engaged and the pilots are not actively engaged with flying the aircraft they’re at altitude and have significant amounts of time to react. The few events that do require immediate responses have extremely constrained responses: essentially push down or pull up (ground proximity, stall, or TCAS alerts) - none of which is the case for road vehicles. The time on a road vehicle between something going wrong, and impact, is generally less 3 seconds (I recall Tesla’s giving 2 seconds of notice). In that time the now driver has to become aware of the change in state, develop situational awareness, and then start to react. Then the actual correct course of action has to actually complete, which also takes time. IMO self driving system that hands off control of the vehicle less than 10s prior to an accident occurring that it is responsible for is the fault of the manufacturer. Obviously a self driving system can’t be held responsible for an accident caused by another vehicle. If the manufacturer’s self driving system is unable to recover from whatever is going on, it’s reasonable to give up and offload to the human in the driver’s seat, but it not reasonable to then say that they were responsible for an accident if they are unable to recover. Again, all these nonsenses trying to avoid being in charge of the car are simply symptoms of the actual problem which is that self driving cars today are unsafe. It’s obviously fairly awful of those people to knowingly circumvent safety systems of something that they are aware is already unsafe, but it’s the manufacturers fault for selling a product that is unsafe in the first place.

dilyevsky 3y ago
s/self driving car people/tesla and comma.ai people/
GianFabien 3y ago
It is instructive to note that airline pilots are very well trained, constantly undertake refresher training and testing. So the competence level of airline pilots is far greater than the average car driver. Yet, there have been many instances when pilots lulled by the monotony of monitoring autopilot piloted planes have failed to respond correctly when the autopilot disengaged with a warning. Managers and development teams of all self-driving technology firms should be required to watch the entire series of "Air Crash Investigations" and actively learn from the decades of experience. Of course, that will never happen. It would contradict their sense of superiority and incessant pursuit of share portfolio gains.
- ckdarby 3y ago
  There's an easy solution but would face extreme opposition though. I think it is hard to argue it would help which is, every driver regardless of situation needs to do a practical refresh class. There are ways of making this optional without the uprise of critics who will argue this is some government control. I'd suggest making it optional but having the data available to insurance companies. If it is helpful and reduces the risk of accidents the insurance companies will be the first to adjust rates for those who don't do it. While there could be an argument that this is some sort of inequality of wealth being able to pay their way out that already exists. I'd rather this not be optional but know it'll be drowned out by government in your life banter.
  - olliej 3y ago
    The problem we’re discussing isn’t “not paying attention while driving”, it’s attention fatigue bought on by requiring constant focus but not actually requiring any actions to be taken. The former is “it turned at an intersection without checking for obstacles” which is driver error. The latter is something that we know objectively that people cannot do. For the non-car cases where this occurs we can generally have many factors to ensure a safe outcome even if an emergency does occur when the operators is distracted. For aircraft the emergency outcomes basically have one of two responses, or there is time for the pilot to regain situational awareness (it takes a lot of time for a plane to go from altitude to sea level, for every other emergency the pilot is trained and commercial aircraft often explicitly state the operation to perform - go up or go down. Trains are spaced so that a train can always just apply emergency brakes, and the driver(engineer?) also knows to just brake as well. The problem with the self driving “attention” requirement is because they have chosen not to handle “emergency/failure occurs while driver is not paying attention”, instead they’re saying “you must always pay attention despite that demand being physiologically implausible” which means you get two outcomes: either the product does not work, or you get crashes from self drive disengaging without warning. The former results in people trying to hack around a faulty product, the latter is a liability shift “if the driver was paying attention as required this crash wouldn’t have happened”.
kbos87 3y ago
You’ve conveniently ignored the fact that the average driver in 2023 (without any help from a driver assistance system) is severely distracted. Surpassing the level of attention and responsiveness of an average driver is a very low bar right now. Also, your focus on the time given to take over before a collision makes little sense. If the vehicle senses a dangerous situation, it doesn’t just throw up its hands and disengage - it’s going to attempt to stop or maneuver out of the way of danger the same way a human would, and people don’t demonstrate a lot of skill or success in those situations either.
- heavenlyblue 3y ago
  Where is the proof of your statement?
  - kbos87 3y ago
    My comment relies partly on common sense (few people would likely argue that drivers are, on average, frequently distracted by their phones) and partly on my own experience having driven using Tesla FSD for thousands of miles. Forced disengagements are very, very rare and the vehicle doesn’t just stop attempting to steer and brake itself.
- cvadict 3y ago
  > it doesn’t just throw up its hands and disengage - it’s going to attempt to stop or maneuver out of the way of danger Lol. That is EXACTLY what every modern auto driving system I've used does. If it gets confused it beeps and then immediately bitches out.
jmpman 3y ago
“It’s obviously fairly awful of those people to knowingly circumvent safety systems of something that they are aware is already unsafe” - you have never driven through west Texas. Straight, flat, perfect visibility for miles, and very little traffic. As long as you’re not driving into the sunrise or sunset, Tesla can handle it well.
- olliej 3y ago
  Tesla advertises that they can handle vastly more than that, and allows you to use their self driving in those different environments. The issue isn't that there are situations it can handle. The problem is that it routinely cannot, and the "driver must always be fully aware and ready to take control with no advance warning" is used purely to shift liability for failings in their own product. I guess a good equivalent would be to imagine you have ABS brakes, but then one day you have to brake suddenly, and the ABS fails and they lock. The Tesla model would be to say "if the ABS system fails during emergency braking you are required to take over, and you can't sue us if this happens and causes an accident"

by olliej 3y ago

dilyevsky 3y ago

s/self driving car people/tesla and comma.ai people/

GianFabien 3y ago

It is instructive to note that airline pilots are very well trained, constantly undertake refresher training and testing. So the competence level of airline pilots is far greater than the average car driver. Yet, there have been many instances when pilots lulled by the monotony of monitoring autopilot piloted planes have failed to respond correctly when the autopilot disengaged with a warning. Managers and development teams of all self-driving technology firms should be required to watch the entire series of "Air Crash Investigations" and actively learn from the decades of experience. Of course, that will never happen. It would contradict their sense of superiority and incessant pursuit of share portfolio gains.

ckdarby 3y ago
There's an easy solution but would face extreme opposition though. I think it is hard to argue it would help which is, every driver regardless of situation needs to do a practical refresh class. There are ways of making this optional without the uprise of critics who will argue this is some government control. I'd suggest making it optional but having the data available to insurance companies. If it is helpful and reduces the risk of accidents the insurance companies will be the first to adjust rates for those who don't do it. While there could be an argument that this is some sort of inequality of wealth being able to pay their way out that already exists. I'd rather this not be optional but know it'll be drowned out by government in your life banter.
- olliej 3y ago
  The problem we’re discussing isn’t “not paying attention while driving”, it’s attention fatigue bought on by requiring constant focus but not actually requiring any actions to be taken. The former is “it turned at an intersection without checking for obstacles” which is driver error. The latter is something that we know objectively that people cannot do. For the non-car cases where this occurs we can generally have many factors to ensure a safe outcome even if an emergency does occur when the operators is distracted. For aircraft the emergency outcomes basically have one of two responses, or there is time for the pilot to regain situational awareness (it takes a lot of time for a plane to go from altitude to sea level, for every other emergency the pilot is trained and commercial aircraft often explicitly state the operation to perform - go up or go down. Trains are spaced so that a train can always just apply emergency brakes, and the driver(engineer?) also knows to just brake as well. The problem with the self driving “attention” requirement is because they have chosen not to handle “emergency/failure occurs while driver is not paying attention”, instead they’re saying “you must always pay attention despite that demand being physiologically implausible” which means you get two outcomes: either the product does not work, or you get crashes from self drive disengaging without warning. The former results in people trying to hack around a faulty product, the latter is a liability shift “if the driver was paying attention as required this crash wouldn’t have happened”.

kbos87 3y ago

You’ve conveniently ignored the fact that the average driver in 2023 (without any help from a driver assistance system) is severely distracted. Surpassing the level of attention and responsiveness of an average driver is a very low bar right now. Also, your focus on the time given to take over before a collision makes little sense. If the vehicle senses a dangerous situation, it doesn’t just throw up its hands and disengage - it’s going to attempt to stop or maneuver out of the way of danger the same way a human would, and people don’t demonstrate a lot of skill or success in those situations either.

heavenlyblue 3y ago
Where is the proof of your statement?
- kbos87 3y ago
  My comment relies partly on common sense (few people would likely argue that drivers are, on average, frequently distracted by their phones) and partly on my own experience having driven using Tesla FSD for thousands of miles. Forced disengagements are very, very rare and the vehicle doesn’t just stop attempting to steer and brake itself.
cvadict 3y ago
> it doesn’t just throw up its hands and disengage - it’s going to attempt to stop or maneuver out of the way of danger Lol. That is EXACTLY what every modern auto driving system I've used does. If it gets confused it beeps and then immediately bitches out.

jmpman 3y ago

“It’s obviously fairly awful of those people to knowingly circumvent safety systems of something that they are aware is already unsafe” - you have never driven through west Texas. Straight, flat, perfect visibility for miles, and very little traffic. As long as you’re not driving into the sunrise or sunset, Tesla can handle it well.

olliej 3y ago
Tesla advertises that they can handle vastly more than that, and allows you to use their self driving in those different environments. The issue isn't that there are situations it can handle. The problem is that it routinely cannot, and the "driver must always be fully aware and ready to take control with no advance warning" is used purely to shift liability for failings in their own product. I guess a good equivalent would be to imagine you have ABS brakes, but then one day you have to brake suddenly, and the ABS fails and they lock. The Tesla model would be to say "if the ABS system fails during emergency braking you are required to take over, and you can't sue us if this happens and causes an accident"