Your visualization helped me understand this! If the center of the distribution is far from P, then all the lines from P to the points in your cluster are basic

Your visualization helped me understand this! If the center of the distribution is far from P, then all the lines from P to the points in your cluster are basically parallel, and you just shift your point cluster which doesn’t help your estimate. But if P is close to the mean, then it sits near the middle of your cluster, so pulling all points towards P is “shrinking” the cluster more than “shifting” it.