3 ms·
Suppose there are a bunch of points on a number line (elevators in the example) and you are standing somewhere between them. You decide to minimize the sum of s
by wikfwikf 4y ago
Suppose there are a bunch of points on a number line (elevators in the example) and you are standing somewhere between them. You decide to minimize the sum of squared differences by the greedy algorithm - make a small move either left or right, whichever makes it smaller, then re-evaluate and repeat.
First just consider one point, A. The squared distance from that point is a parabola with its apex at A, or as a formula (x - A)^2. The derivative of that is 2x - 2A. So if 2x - 2A > 0, moving to the right makes the squared distance go up, moving to the left makes it go down. If it's < 0 then the opposite is true. If it equals 0 then you have minimized the squared distance to A. All this is trivial, in the case of a single point, because x = A is obviously the solution (the mean of a single value is that value).
But you want to minimize the sum of the squared differences. So you are checking sum(2x - 2Ai) where Ai are all the different points. This means comparing 2nx with sum(2Ai), if n is the number of points. Or equivalently, you can divide by 2n and compare x with sum(Ai)/n. This formula is just the mean of the points. If this is greater than x, you should move a bit to the right. If it is smaller, you should move to the left. If equal, you have minimized the sum of squared differences.
We didn't prove uniqueness but we've established that the derivative of the sum of squared distances has only linear terms in it, so a little more calculus will easily allow us to check that.
tl;dr: To minimize a function requires an equation in that function's derivative. To minimize sum of squared errors we solve an equation with only linear terms.