5 ms·

Is there any intuitive explanation for why the mean minimizes the squared error? I know it IS true - this property is used for linear regressions etc - but I co

• shikoba 4y ago
  If you know about inertia, the grabiy center is where inertia is minimized. But it's just a way to rephrase it.
• GuB-42 4y ago
  Distances are squares, actually square roots of squares, in Euclidian space, so it makes sense to find squares when we deal with distances. An intuitive way to look at it is that square function is the simplest function that goes down until it reaches a minimum, and then back up again. If you want this property, you can expect to find squares somewhere. Distance have this property: if you move in a straight line, the distance to your target will go down as you are closing in, then you reach it, then if goes back up again after you pass it. That's the general intuition for where the squares can be. Let's be a bit more formal. Let's flip the problem and try to minimize the sum of the squared distances (or squared error, same thing). In 1D, the sum of the squared distances to a and b is (a-x)^2 + (b-x)^2 = a^2 - 2ax + x^2 + b^2 - 2bx + x^2 = a^2 + b^2 - 2(a+b)x + 2x^2 It is a parabola, the minimum is where the derivative is zero. The derivative is -2(a+b) + 4x, solve -2(a+b) + 4x = 2 for x and you have x = (a+b)/2, which is your average. Of course, it works with more than 2 distances. Another intuition: to minimize something, usually, we take a derivative, and when we take the derivative of something squared, the square tends to disappear, so it shouldn't surprising that the minimum of something squared has no squares in it.
  • dragonwriter 4y ago
    > An intuitive way to look at it is that square function is the simplest function that goes down until it reaches a minimum, and then back up again By what measure of simplicity is x^2 simpler than abs(x)? I mean, if you added the caveat “with a continuous derivative”, sure.
    • orangecat 4y ago
      By what measure of simplicity is x^2 simpler than abs(x)? Its description is shorter in English and in most programming languages, mainly because it doesn't have a conditional. (Which is related to it being differentiable everywhere).
• wikfwikf 4y ago
  Suppose there are a bunch of points on a number line (elevators in the example) and you are standing somewhere between them. You decide to minimize the sum of squared differences by the greedy algorithm - make a small move either left or right, whichever makes it smaller, then re-evaluate and repeat. First just consider one point, A. The squared distance from that point is a parabola with its apex at A, or as a formula (x - A)^2. The derivative of that is 2x - 2A. So if 2x - 2A > 0, moving to the right makes the squared distance go up, moving to the left makes it go down. If it's < 0 then the opposite is true. If it equals 0 then you have minimized the squared distance to A. All this is trivial, in the case of a single point, because x = A is obviously the solution (the mean of a single value is that value). But you want to minimize the sum of the squared differences. So you are checking sum(2x - 2Ai) where Ai are all the different points. This means comparing 2nx with sum(2Ai), if n is the number of points. Or equivalently, you can divide by 2n and compare x with sum(Ai)/n. This formula is just the mean of the points. If this is greater than x, you should move a bit to the right. If it is smaller, you should move to the left. If equal, you have minimized the sum of squared differences. We didn't prove uniqueness but we've established that the derivative of the sum of squared distances has only linear terms in it, so a little more calculus will easily allow us to check that. tl;dr: To minimize a function requires an equation in that function's derivative. To minimize sum of squared errors we solve an equation with only linear terms.