3 ms·
The thing about humans is we're relatively good at understanding our own heuristics, and rating the confidence of those beliefs. If there's a storm we reduce di
by fwip 8y ago
The thing about humans is we're relatively good at understanding our own heuristics, and rating the confidence of those beliefs. If there's a storm we reduce distractions and pay more attention to the road, drive more conservatively. Other passengers will understand not to distract the driver.
But if the car is confused by some occasional paint splotches on the road that clearly aren't lane markings to our human eyes, we don't have any understanding that the car is being misled. Like that video the other day of the Tesla clipping the barrier - up until about a half second before impact I would have assumed it knew what it was doing and was on track to avoid it.
It's unrealistic to expect a human driver to take over immediately in failure scenarios that the person can't recognize.
- Robotbeat 8y agoI think there's lots of work to be done in quantifying and comparing discrepencies and lack of confidence in neural net classifications and sensor/model outputs. Part of this is human interface design. An example of this is the "sensor disagree" light on the Boeing 737 max (which unfortunately was optional equipment not installed for the Ethiopian Airlines 737 Max crash), although I do believe we can do much better than a mere light. A full screen and a sound system with over-the-air updates means we ought to be able to field a really good (and continuously improving) system for communicating when the autopilot's sensors disagree or if the model has low confidence. This is likely going to take years or even decades of operational use over millions of vehicles to refine with good confidence.