Yes! Lots of them actually. 1. The gaussian distribution is important (all sums converge to it) and the squared difference from the mean characterizes it. 2.

Yes! Lots of them actually. 1. The gaussian distribution is important (all sums converge to it) and the squared difference from the mean characterizes it. 2. Euclidean space is important, and squared errors stay the same if you rotate everything. (Other errors don't). 3. Linear regression with squared error has a closed form solution. Other types of models converge very fast when using it because the further you are from optimal, the bigger your gradient is.