That sounds like it would end up with us having a situation where every time there is a human on the path, the car needs to slow down to 5mph in case it decides

That sounds like it would end up with us having a situation where every time there is a human on the path, the car needs to slow down to 5mph in case it decides to jump out into the road. A human can tell if there is something going on at the side of the road that might make the human jump into the road. Machines are nowhere near being able to understand the context of many situations that allow humans to predict these things.