I think part of it comes down to reliability of the sensors - if you detect an object 200m down the road, 5 metres to the right and you have only a small error

I think part of it comes down to reliability of the sensors - if you detect an object 200m down the road, 5 metres to the right and you have only a small error in the reading, you could easily get the impression that it's moving into the path of the car. However, if you can identify what it is then you can identify if it's likely that it really is going to move into your path. These sensors really aren't as perfect as you would hope which is why so much effort is being put into machine intelligence.