Even if you can identify the hand signal, and identify that a construction worker is giving them (establishing authority that the car should follow this signal)

Even if you can identify the hand signal, and identify that a construction worker is giving them (establishing authority that the car should follow this signal), the signal contains a lot of implied information. If the worker indicates you to go into the opposing lane, how long should the car remain there? How do you tell the construction part is done and you should return to normal self-driving logic? Sometimes there are cones, but sometimes not. American workers do often have Slow / Stop signs, giving a clear indication of when to move at least. If you are going down a street slowly, and someone is wearing a hi-vis vest for their own safety, and they point at something, is your car going to follow that pedestrian's instructions?