4 ms·

It's bound to happen, but also likely to be the result of the driver not paying attention.

granzymes 6y ago
Right, but if the car does it right 99% of the time it's just human nature to stop paying attention. The article makes a great point that slowly improving the autopilot with a person behind the wheel is super dangerous because we get lulled into a false sense of security. I hope Tesla just keeps it to highway lane keeping until they're ready to roll out better sensors.
- mortenjorck 6y ago
  This principle is glaringly obvious in retrospect, and seeing it put so succinctly actually changed my mind. I previously thought Tesla's approach was an achievable long shot, but I'm now convinced it's fundamentally wrong. The safety-driver model pursued by Waymo and every other autonomy startup is the only responsible way to train an autonomy network because the drivers are specifically trained to look out for edge cases, versus consumers who will easily be lulled into complacency once the system looks superficially like it's running at level-5 competency.
  - dragonwriter 6y ago
    While I agree that the Waymo, et al., way is the only responsible way, that is not an argument against Tesla’s way being workable, just that it is grossly unethical.
ra7 6y ago
Krafcik talks about this very issue in the article on how driver assistance systems give false sense of security to the driver. > The Waymo team believes its own early experience—when it was the Google self-driving car project—bears that out. In the early 2010s, Google developed a driver-assistance system similar to today's Autopilot and considered selling it to automakers. But when they let Google employees test the software on public roads, they found that drivers came to trust it way too quickly. Drivers who were supposed to be closely monitoring the system instead spent their time looking at their phones, putting on makeup, and other distractions. > The fundamental challenge here is that the better a driver-assistance system gets, the harder it is to get drivers to pay attention, and the less likely they are to be prepared if the software makes a mistake. The Google team didn't see a good solution to this problem, so they completely changed their strategy. They focused on building a self-driving taxi service that would never have customers in the driver's seat, relying on trained, professional safety drivers to oversee the software during testing.
nightcracker 6y ago
Can you pay attention to a video game for two hours, and during that time react to some important event within 1 second reaction time? I bet you can. Can you do the same when staring at a wall for two hours?
mxschumacher 6y ago
coffee cups say "be careful this is hot", but the regulator allows a product called "Full Self Driving" to be sold - I think ultimately consumer protection will step in here like it has in Germany [0] 0: https://www.reuters.com/article/us-tesla-autopilot-germany/german-court-bans-tesla-ads-claiming-its-cars-are-capable-of-autonomous-driving-idUSKCN24F1T5 https://www.reuters.com/article/us-tesla-autopilot-germany/g...
- breakfastduck 6y ago
  Yeah it's a fantastic example of technological developments far outpacing legislation

Right, but if the car does it right 99% of the time it's just human nature to stop paying attention. The article makes a great point that slowly improving the autopilot with a person behind the wheel is super dangerous because we get lulled into a false sense of security. I hope Tesla just keeps it to highway lane keeping until they're ready to roll out better sensors.

mortenjorck 6y ago
This principle is glaringly obvious in retrospect, and seeing it put so succinctly actually changed my mind. I previously thought Tesla's approach was an achievable long shot, but I'm now convinced it's fundamentally wrong. The safety-driver model pursued by Waymo and every other autonomy startup is the only responsible way to train an autonomy network because the drivers are specifically trained to look out for edge cases, versus consumers who will easily be lulled into complacency once the system looks superficially like it's running at level-5 competency.
- dragonwriter 6y ago
  While I agree that the Waymo, et al., way is the only responsible way, that is not an argument against Tesla’s way being workable, just that it is grossly unethical.

ra7 6y ago

Krafcik talks about this very issue in the article on how driver assistance systems give false sense of security to the driver. > The Waymo team believes its own early experience—when it was the Google self-driving car project—bears that out. In the early 2010s, Google developed a driver-assistance system similar to today's Autopilot and considered selling it to automakers. But when they let Google employees test the software on public roads, they found that drivers came to trust it way too quickly. Drivers who were supposed to be closely monitoring the system instead spent their time looking at their phones, putting on makeup, and other distractions. > The fundamental challenge here is that the better a driver-assistance system gets, the harder it is to get drivers to pay attention, and the less likely they are to be prepared if the software makes a mistake. The Google team didn't see a good solution to this problem, so they completely changed their strategy. They focused on building a self-driving taxi service that would never have customers in the driver's seat, relying on trained, professional safety drivers to oversee the software during testing.

nightcracker 6y ago

Can you pay attention to a video game for two hours, and during that time react to some important event within 1 second reaction time? I bet you can. Can you do the same when staring at a wall for two hours?

mxschumacher 6y ago

coffee cups say "be careful this is hot", but the regulator allows a product called "Full Self Driving" to be sold - I think ultimately consumer protection will step in here like it has in Germany [0] 0: https://www.reuters.com/article/us-tesla-autopilot-germany/german-court-bans-tesla-ads-claiming-its-cars-are-capable-of-autonomous-driving-idUSKCN24F1T5 https://www.reuters.com/article/us-tesla-autopilot-germany/g...

breakfastduck 6y ago
Yeah it's a fantastic example of technological developments far outpacing legislation