4 ms·

It is amazing how many statisticians are still using pseudo-inverses in numeric computation. What is specifically wrong with the pseudo-inverse? Is it that eig

It is amazing how many statisticians are still using pseudo-inverses in numeric computation. What is specifically wrong with the pseudo-inverse? Is it that eigenvalues close to zero (that might actually be zero, but are not due to FP error) get magnified immensely? Or is it the general principle that inverting a matrix is usually a bad idea, for reasons that John D. Cook elaborates quite well? R. It is an atrocious monstrocity. Perhaps statisticians can live with it. Fine, but don't expect good programmers coming out of using R for five years. Python would be far better. Python has great libraries but I could see Clojure overtaking it as the data science language in the next 10-15 years, and Haskell has some obvious strengths for production systems (strong, static typing).

zaphar 12y ago
I would actually bet on Julia before Clojure as the next big data science language.
east2west 12y ago
The following quote is from "Matrix analysis and applied linear algebra" by Carl D Meyer, p 423-424. "Generalized inverses are useful in formulating theoretical statements as those above, ut, just as in the case of the ordinary inverse, generalized inverses are not practical computational tools. In addition to being computationally inefficient, serious numerical problems result from the fact that A+ needs not be a continuous function of the entries of A. ... Not only is A+(x) discontinuous in the sense that lim_{x->0}A+(x)!=A+(0), but it is discontinuous in the worst way because as A(x) comes closer to A(0) the matrix A+(x) moves farther away from A+(0)." Here A+ denotes Moore-Penrose pseudo-inverse and A(x) = [1, 0; 0, x] in Matlab notation (or Numpy? I am very rusty on Matlab). And in case you are wondering why the generalized inverse is plural, there are different kinds of generalized inverses, Drazin inverse being a well defined alternative.
- thearn4 12y ago
  Pseudo-inverse (and other direct methods) do have a lot going against them, especially as problems scale. In practice, iterative methods introduce their own challenges as well though. Optimal algorithm selection is highly problem dependant, and often hinges on knowing how to generate an appropriate preconditioner. GMRES is still pretty much my favorite general method for inverse problems, but it comes with plenty of nuances that you really don't see when using a direct method on a tiny problem that you can bank on being well-conditioned. It's hard to say how often those kinds of problems exist in the real world (in my case: exceedingly rare), but they are still out there.

zaphar 12y ago

I would actually bet on Julia before Clojure as the next big data science language.

east2west 12y ago

The following quote is from "Matrix analysis and applied linear algebra" by Carl D Meyer, p 423-424. "Generalized inverses are useful in formulating theoretical statements as those above, ut, just as in the case of the ordinary inverse, generalized inverses are not practical computational tools. In addition to being computationally inefficient, serious numerical problems result from the fact that A+ needs not be a continuous function of the entries of A. ... Not only is A+(x) discontinuous in the sense that lim_{x->0}A+(x)!=A+(0), but it is discontinuous in the worst way because as A(x) comes closer to A(0) the matrix A+(x) moves farther away from A+(0)." Here A+ denotes Moore-Penrose pseudo-inverse and A(x) = [1, 0; 0, x] in Matlab notation (or Numpy? I am very rusty on Matlab). And in case you are wondering why the generalized inverse is plural, there are different kinds of generalized inverses, Drazin inverse being a well defined alternative.

thearn4 12y ago
Pseudo-inverse (and other direct methods) do have a lot going against them, especially as problems scale. In practice, iterative methods introduce their own challenges as well though. Optimal algorithm selection is highly problem dependant, and often hinges on knowing how to generate an appropriate preconditioner. GMRES is still pretty much my favorite general method for inverse problems, but it comes with plenty of nuances that you really don't see when using a direct method on a tiny problem that you can bank on being well-conditioned. It's hard to say how often those kinds of problems exist in the real world (in my case: exceedingly rare), but they are still out there.