2 ms·

By human factors I mean people's interaction with and dependency on technology. As example say Tesla FSD works great on the CA highways and people start to dep

By human factors I mean people's interaction with and dependency on technology. As example say Tesla FSD works great on the CA highways and people start to depend on it. Maybe they starting scrolling through instagram on regular basis on their commute. Then they decide to go to Mendocino up whatever that super twisty and scary road is called. The put on FSD and the Tesla does a great job. They start to depend on the Tesla to do even the more technical driving. So they stop paying attention even on the harder driving sections. Six months down the line they are using the FSD drive on a twisty section of road. The sun is the eyes of vision based FSD and there are a bunch of cyclists in the road. The FSD hasn't been trained on this case (or hits a bug or whatever) and doesn't detect the cyclists. The driver who has been "Trained" by the FSD to not pay attention is fucking around with their phone. The tesla plows into the cyclists. This is my fear - the fear that you detrain people on the responsibility of driving. I suppose at the base of this is I have very low trust in how people will reason about their role with technology. People who don't know about the limitations of the machine will assume it is accurate. People will work to exploit the limitations (two hands on the wheel, eye tracking, etc). If this technology was marked as "crash avoidance" technology I'd be much less worried. To take the twisty road example, imagine if the Tesla can not only self drive the car to a safe point in case of drive incapciation - that's pretty awesome but also puts the clear responsibility on the human that they are doing the driving.

ajross 5y ago
> Then they decide to go to Mendocino Again, that's a multi-mode failure. You're assuming that (1) edge cases must exist somewhere where FSD is unsafe, (2) that drivers will find them, and (3) that when they do, the existing supervised beta paradigm won't work. In isolation any of those can happen, sure. But in combination they're vanishing. Broadly you're doing the luddite/reactionary thing and demanding perfection: as long as you can conjure a scenario in your head where something "might" happen you want to act as if it "will" and demand that we refuse to implement new solutions. But the goal isn't perfection and never has been, for the simple reason that the existing solution also sucks really bad. Real human drivers in Mendocino with the sun in their eyes hit bikers too!
- harshaw 5y ago
  To be a bit more obvious: I am afraid. I am not telling other people to be afraid. But I am very curious about the general public's view of this technology.

ajross 5y ago

> Then they decide to go to Mendocino Again, that's a multi-mode failure. You're assuming that (1) edge cases must exist somewhere where FSD is unsafe, (2) that drivers will find them, and (3) that when they do, the existing supervised beta paradigm won't work. In isolation any of those can happen, sure. But in combination they're vanishing. Broadly you're doing the luddite/reactionary thing and demanding perfection: as long as you can conjure a scenario in your head where something "might" happen you want to act as if it "will" and demand that we refuse to implement new solutions. But the goal isn't perfection and never has been, for the simple reason that the existing solution also sucks really bad. Real human drivers in Mendocino with the sun in their eyes hit bikers too!

harshaw 5y ago
To be a bit more obvious: I am afraid. I am not telling other people to be afraid. But I am very curious about the general public's view of this technology.