3 ms·
Or, equivalently, you can do the "engineer's trick": To solve an over-determined system Ax=b, where A is a rectangular matrix, you multiply both sides by Aᵀ.
by enriquto 4y ago
Or, equivalently, you can do the "engineer's trick": To solve an over-determined system Ax=b, where A is a rectangular matrix, you multiply both sides by Aᵀ. Thus you obtain a new system whose matrix AᵀA is square, and you can solve normally.
- mkl 4y agoThat doesn't explain why it's least squares though, which was FabHK's point. In isolation it actually seems like a pretty arbitrary thing to do.
- enriquto 4y agoSure, it's mostly a mnemonic. But you can do it easily in your head, even in infinite dimensions. For example, if you recall that (due to stokes theorem) the divergence is minus the transpose of the gradient, you can solve many variational problems that way. The canonical example is: find a function u whose gradient is a given vector field F. This is an over-determined system without a solution. Applying the above trick, you compute the divergence of both sides to obtain Poisson equation Δu=div(F), that you can solve. This is equivalent to finding the least-square minimizer of the energy E(u)=∫|∇u-F|²