I think in this case, the fundamental problem that other replies also stated is the lack of a generality of context. The system was trained to recognize balls,

I think in this case, the fundamental problem that other replies also stated is the lack of a generality of context. The system was trained to recognize balls, but it had no concept of a person. Train it on balls and people, and I'm sure the performance would improve because you can tell the system to not track things classified as a person. But then your system is also trying to draw boxes around people. So, now maybe your BB (bounding box) for people and ball keep overlapping. Now, you train it on image segmentation (draw an outline around objects) as opposed to bounding box selection and the performance also improves. The downside is now you require significantly more compute power on hand/lower sampling rate, etc. I think it is a testament to not just human complexity, but human efficiency.