Yes, now we can also do best L^1 approximations. IIRC -- I'm in a hurry this morning -- it's a linear programming problem or some such but I haven't thought abo

Yes, now we can also do best L^1 approximations. IIRC -- I'm in a hurry this morning -- it's a linear programming problem or some such but I haven't thought about that in decades. You bring up a lot of stuff I've never reviewed. > but that does not mean it is a suitable performance metric to use. If the plane do the L^2 projection on is really close, then the error is really small, and what's not to like? Really close is good enough for me. Right, there are three biggie choices, L^1 (minimize the sum of absolute values of the errors), L^2 (minimize the sum of squared errors), and L^infinity (minimize the worst error). L^2's fine with me, good enough for government work, okay first cut, day in and day out. If in some particular case L^2 has some problems, then maybe something like regression is not the right tool for the problem. Gee, we do a lot of L^2: We get orthogonal components so that we can get L^2 cafeteria style, just pick the components we want. E.g., in filtering stochastic processes, we take a Fourier transform -- that is finding the coefficients of the sample path on the sine-cosine orthogonal components. Or we do a convolution, which is the same thing in the end. The approximation we get is an L^2 approximation. So, with JPG -- sure, it's L^2. Right, JPG does funny stuff nearly lines. Life's not perfect! For some things, sure we want L^infinity: E.g., if we want to use a quotient of polynomials to approximate the usual special functions, then we want to minimize the worst error and do what is called Chebyshev approximation, but this is very specialized.