I think in this case, from the video, it is clear that the safety driver is not looking at the road all the time. But lets consider the scenario that the human

I think in this case, from the video, it is clear that the safety driver is not looking at the road all the time. But lets consider the scenario that the human driver was 100% alert and focused and were looking at the road all of the time. How would this have played out in this scenario? The car is driving along. The human driver sees the pedestrian 20-30m down the road but notices that the car is not slowing down. What does the human driver do in this situation? Slam on the brakes just in case? Do nothing assuming the car has worked out that it is safe? Or wait assuming that the car will make a decision in time, and then slam on the brakes if it doesn't ... but would it be too late by then? What (if any) decision do you make to override - and when do you make it - when you're using an autonomous system? I dont think this sort of thing is clear-cut. Obviously in hindsight in this scenario we'd all pick "slam on the brake", but in this sort of scenario where it is kinda "now or never" for braking, if you override the system too early you never get to really test/develop the system, but if you leave it too late you dont have enough time to prevent the accident. There must be an incentive/encouragement for these human drivers to "let the system do its job" rather than disengaging all the time, otherwise no one would ever let the systems run in full-auto all the time. It sounds like a tough job. Presumably there are systems in the car to indicate what it "sees", but cross-referencing that against what your eyes see is probably quite difficult "Of the 50-60 things on the screen that the car has registered, it has missed this one thing." I'd not want that job.