This seems like a misunderstanding of the problem. The problem is that it's very difficult paying attention to something you have no control over or interaction

This seems like a misunderstanding of the problem. The problem is that it's very difficult paying attention to something you have no control over or interaction with. To stick in a prod to punish people isn't going to make it easier. You'll end up just firing a lot of drivers. What would be better is to give the human something to do. For example you could put a set of input devices in the car for the human: Tell them to record their 'level of confidence' so you can check that against the cars confidence levels, or input suggesting steering adjustments to gauge where the car is differing from normal human behaviour. That way when your driver has become an active participant (albeit probably useless) in the driving.