Distances are squares, actually square roots of squares, in Euclidian space, so it makes sense to find squares when we deal with distances. An intuitive way to

Distances are squares, actually square roots of squares, in Euclidian space, so it makes sense to find squares when we deal with distances. An intuitive way to look at it is that square function is the simplest function that goes down until it reaches a minimum, and then back up again. If you want this property, you can expect to find squares somewhere. Distance have this property: if you move in a straight line, the distance to your target will go down as you are closing in, then you reach it, then if goes back up again after you pass it. That's the general intuition for where the squares can be. Let's be a bit more formal. Let's flip the problem and try to minimize the sum of the squared distances (or squared error, same thing). In 1D, the sum of the squared distances to a and b is (a-x)^2 + (b-x)^2 = a^2 - 2ax + x^2 + b^2 - 2bx + x^2 = a^2 + b^2 - 2(a+b)x + 2x^2 It is a parabola, the minimum is where the derivative is zero. The derivative is -2(a+b) + 4x, solve -2(a+b) + 4x = 2 for x and you have x = (a+b)/2, which is your average. Of course, it works with more than 2 distances. Another intuition: to minimize something, usually, we take a derivative, and when we take the derivative of something squared, the square tends to disappear, so it shouldn't surprising that the minimum of something squared has no squares in it.