While I agree with the concern, I think there's almost zero chance that a neural net trained on people looking at phones would classify a driver looking at hand

While I agree with the concern, I think there's almost zero chance that a neural net trained on people looking at phones would classify a driver looking at hand controls as that same thing. The neural net would more likely have another classification of "unknown attention", being unable to conclusively classify the driver's attention.