3 ms·

I can't name any applications off the top of my head, other than iterative matrix multiplication for approximate eigenvector finding in square matrixes. But I d

I can't name any applications off the top of my head, other than iterative matrix multiplication for approximate eigenvector finding in square matrixes. But I don't know what's actually used for finding eigenvectors (or other decompositions for that matter).

TimorousBestie 1y ago
It’s a fairly common operation. The sample covariance of a vector-valued random variable is XX^t/N.
bee_rider 1y ago
Could be useful when doing an SVD as well (although, really, you don’t want X’X but X’Xv for some vector…).
vqv 1y ago
It’s pretty fundamental in a variety of multivariate statistical methods. If the rows of X are multi variate observations, then XX’ is the Gram matrix (of dot products). This can be used in clustering and regression. If the columns of X are (centered) multivariate observations, then XX’ is a scalar multiple of the sample covariance matrix. This is used in PCA. But in large scale applications you may not want to store XX’ but instead are interested in computing products of the form XX’ v on the fly.
- adgjlsfhk1 1y ago
  generally good implementations use QR instead
constantcrying 1y ago
I think one particularly interesting result is that superior algorithms for numerical linear algebra exist at all and can be found by artificial intelligence. XX^T is the matrix of all piecewise dot products of vectors in X and as others have pointed out there are legitimate applications. Another one being e.g. transforming an undetermined linear system into a least squares problem.
- FabHK 1y ago
  The solution to the least squares problem Ax ≈ b is given by x = (A'A)^-1 A'b, but modern solvers never form the matrix product A'A explicitly. Even for the SVD it isn't formed.
- jerf 1y ago
  Google put something out like this recently too. The original is https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/ https://deepmind.google/discover/blog/alphaevolve-a-gemini-p..., but it's sort of buried in there. Here's the YouTube video that introduced it to me, forwarded to the spot where the host talks about the improved 4x4 matrix multiplication algorithm: https://www.youtube.com/watch?v=sGCmu7YKgPA&t=396s https://www.youtube.com/watch?v=sGCmu7YKgPA&t=396s It's only a slight tweak, but it's a slight tweak to something pretty highly studied and still impressive.
blobbers 1y ago
Covariance.
Bootvis 1y ago
I expect this algorithm or similar to work for X^tX as well. Fun fact, that operation is common enough that a trading firm was named after it: XTX Markets.

TimorousBestie 1y ago

It’s a fairly common operation. The sample covariance of a vector-valued random variable is XX^t/N.

bee_rider 1y ago

Could be useful when doing an SVD as well (although, really, you don’t want X’X but X’Xv for some vector…).

vqv 1y ago

It’s pretty fundamental in a variety of multivariate statistical methods. If the rows of X are multi variate observations, then XX’ is the Gram matrix (of dot products). This can be used in clustering and regression. If the columns of X are (centered) multivariate observations, then XX’ is a scalar multiple of the sample covariance matrix. This is used in PCA. But in large scale applications you may not want to store XX’ but instead are interested in computing products of the form XX’ v on the fly.

adgjlsfhk1 1y ago
generally good implementations use QR instead

constantcrying 1y ago

I think one particularly interesting result is that superior algorithms for numerical linear algebra exist at all and can be found by artificial intelligence. XX^T is the matrix of all piecewise dot products of vectors in X and as others have pointed out there are legitimate applications. Another one being e.g. transforming an undetermined linear system into a least squares problem.

FabHK 1y ago
The solution to the least squares problem Ax ≈ b is given by x = (A'A)^-1 A'b, but modern solvers never form the matrix product A'A explicitly. Even for the SVD it isn't formed.
jerf 1y ago
Google put something out like this recently too. The original is https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/ https://deepmind.google/discover/blog/alphaevolve-a-gemini-p..., but it's sort of buried in there. Here's the YouTube video that introduced it to me, forwarded to the spot where the host talks about the improved 4x4 matrix multiplication algorithm: https://www.youtube.com/watch?v=sGCmu7YKgPA&t=396s https://www.youtube.com/watch?v=sGCmu7YKgPA&t=396s It's only a slight tweak, but it's a slight tweak to something pretty highly studied and still impressive.

blobbers 1y ago

Covariance.

Bootvis 1y ago

I expect this algorithm or similar to work for X^tX as well. Fun fact, that operation is common enough that a trading firm was named after it: XTX Markets.