11 ms·
Eigenvectors from eigenvalues
- bhl 7y agoThere's a backstory to this paper from the r/math subreddit [1]. One of the authors initially posted the question there, and stumbling upon a similar question on math overflow emailed Terry Tao. And surprisingly, he responded. [1] https://www.reddit.com/r/math/comments/ci665j/linear_algebra_question_from_a_physicist/ https://www.reddit.com/r/math/comments/ci665j/linear_algebra...
- james_s_tayler 7y agoWow. This is so great.
- leereeves 7y agoNot only responded, but responded in under 2 hours, which is amazing from a busy professor like Tao.
- bcaa7f3a8bbc 7y ago> responded in under 2 hours and gave 3 different proofs!
- mkl 7y agoNot only responded in under 2 hours, but "Tao’s reply also included three independent proofs of the identity." From the article we discussed earlier, https://www.quantamagazine.org/neutrinos-lead-to-unexpected-discovery-in-basic-math-20191113/ https://www.quantamagazine.org/neutrinos-lead-to-unexpected-... https://news.ycombinator.com/item?id=21528425 https://news.ycombinator.com/item?id=21528425
- blt 7y agoit indicates how important this result is!
- dmix 7y agoSomeone replied: > Apparently this relationship was discovered in 2014 by a dutch mathematician or physician. https://arxiv.org/abs/1401.4580 https://arxiv.org/abs/1401.4580 https://old.reddit.com/r/math/comments/ci665j/linear_algebra_question_from_a_physicist/f7j2s2t/ https://old.reddit.com/r/math/comments/ci665j/linear_algebra... I don't know anything about the subject, I'm curious what they mean by that.
- consp 7y agoHe is a professor of telecommunication networks at the Delft University of Technology. see https://www.tudelft.nl/staff/p.f.a.vanmieghem/ https://www.tudelft.nl/staff/p.f.a.vanmieghem/ edit: The apperantly renamed it Network Architecture & Services since I finished my education.
- teilo 7y agoDiscovered, sort of. His method was nearly the same as the one brought to Tao, but he didn't make the connection to the full correlation between eigenvalues and eigenvectors when dealing with real numbers. Tao used the method to explicitly prove this correlation when it was discovered by Denton, Parke and Zhang.
- Ceezy 7y agoThat s great, but any good bachelor students should be able to give you a similar answer.
- lxe 7y agoFor a second I thought this was going to be HN's darling "Eigenvectors AND Eigenvalues" explained visually http://setosa.io/ev/eigenvectors-and-eigenvalues/ http://setosa.io/ev/eigenvectors-and-eigenvalues/
- est31 7y agoSee also prior discussion: https://news.ycombinator.com/item?id=21528425 https://news.ycombinator.com/item?id=21528425
- xxpor 7y agoSo it's been checks notes 9 years since I've had to deal with eigenvectors and eigenvalues. So the post went way over my head, but folks on twitter were making quite a big deal over the post last night when it went up on arXiv. Could someone explain this like I have a computer engineering bachelors degree? :)
- klodolph 7y agoYou might prefer the earlier post, https://news.ycombinator.com/item?id=21528425 https://news.ycombinator.com/item?id=21528425 Basically, this allows you to compute eigenvectors from eigenvalues of minor matrices. It is an interesting identity, but speculating on its applications is beyond my expertise.
- te_platt 7y agoIt's something like an economist finding a $20 dollar bill on the ground and it's actually a real $20 bill that no one has picked up yet.
- Confusion 7y agoFinding a $20 bill on Times Square
- scottlocklin 7y agoMore like a gold sovereign in Piccadilly. I only skimmed, but it's a sweet result, and something an insightful undergrad could have come up with, but never did. Linear algebra for large scale "muh big data" applications has had a mini renaissance in recent years. Haven't seen any this cute and basic though.
- randomsearch 7y agoDon’t know how simple you want me to go, so apologies if required. You’ve got a matrix A, ie a 2D array of numbers. You’ve got a vector v, that is a 1D array of numbers. You can multiply A by v, written Av, to get a new vector. Matrices can therefore be viewed as devices for taking a vector and giving a new one, ie as linear transformations. You can design a matrix to rotate vectors, stretch them etc. An eigenvector of a matrix is a vector that changes only by a constant factor c, when you apply (multiply by) the matrix. So Av = kv where k is a number. For example, if k=2 then applying A just doubles every number in v. Eigenvectors are stretched or squashed by the matrix, no funny business like shearing or rotating. An eigenvalue is the “k” mentioned above, ie there’s at least one eigenvector such that Av=kv. There could be multiple eigenvectors all with the same k, ie all changed in the same simple way by applying the matrix. Ok? Slight complication: matrix A may contain complex numbers, not just your common garden variety real numbers. But the above all still applies. “Symmetric matrices” are matrices that have a symmetry across the top left to bottom right diagonal. So the number at A[i][j] is the same number as that at A[j][i]. “Hermitian” matrices are the equivalent of symmetric matrices when we’re dealing in complex numbers. Rather than A[i][j] being equal to A[j][i], if one entry is a complex number then the other entry is the “complex conjugate” of the first entry. (In case you’ve forgotten complex numbers: we extend the real numbers to including numbers of the form r + zi, where r is the “real part” (just a real number) and “zi” is a real number z multiplied by i, where i = sqrt(-1). Imaginary numbers were introduced to allow us to solve equations we couldn’t otherwise solve, but have turned out to be super useful everywhere. The “complex conjugate” of an imaginary number is obtained by flipping the sign of z, eg (2 + 3i) -> (2 - 3i).) Anyway, so Hermitian matrices are “symmetric” in this way and turn out to be super useful in physics. They also have the property that when you multiply a Hermitian matrix H by a vector v, the possible eigenvalues are real numbers, despite the fact that H may contain complex numbers. What the paper states is a relationship between the eigenvalues and eigenvectors of a Hermitian matrix H and the eigenvalues of a smaller “submatrix” matrix H’, where H’ is just the same as H but with a column and row removed, where the index of the row and column is the same. This is neat, because it can be used to speed up the calculation of eigenvectors (as important as say array sorting in CS) in some specific situations. Commenting further is beyond me.
- auggierose 7y agoThis is a good example of math being presented in an unnecessarily confusing way. Values lambda_i(A) are introduced, but then in the formula also values lambda_i(M) are referenced for some M which is not A. Basically, we have no idea what lambda_i(M) might mean from the statement of the theorem. Is the order lambda_1(M), lambda_2(M) ... related to the order lambda_1(A), lambda_2(A), ... How many lambda_i(M) exist? And so on. Maybe this is all clear if you deal with minor matrices all the time, but when I see a statement like that I am not even interested in its proof anymore.
- banachtarski 7y agoTerry is being abundantly clear. Why are you trying to correlate indices in two separate product series?
- auggierose 7y agoHe is not being "abundantly clear". He doesn't give a definition of lambda_i(M) for the various M appearing in the theorem that are not A.
- _hardwaregeek 7y agoDoesn't he say it here: where {M_j} is the {n-1 \times n-1} Hermitian matrix formed by deleting the {j^{th}} row and column from {A}.
- auggierose 7y agoYes, he says what M_j is, but he does not say what lambda_i(M_j) is.
- tomrod 7y agoOkay, Donald Knuth might be upset that Terry failed to introduce the notion of a spectrum space in a Matrix Analysis paper to an audience that should know about it already. If I tell you f(x) = y, do you get upset that I don't define the domain, range, codomain, notion of a function, and so forth? Or if I tell you that https:// https:// prepends a good portion of URLs on the world-wide web, would you subject it to the same scrutiny? I hope note -- to require such would be unnecessarily limiting. My $0.02.
- banachtarski 7y agoFor those curious about applications: Suppose you have a learning algorithm that requires the eigenvectors/eigenvalues for a large matrix. Suppose new data is streamed in that increase the size of the matrix. well, now you have an algorithm for updating your existing solution instead of needing to compute the entire thing from scratch. (consider updating principal components over time for example). Suppose you want to accelerate the process of computing eigenvectors for a large matrix. Well, you can now do parallel reduction relatively trivially given the theorem. *edit: wording.
- deleted 7y ago[deleted]
- auggierose 7y agoIf the matrix is symmetric (or hermitian in the complex case).
- yxhuvud 7y agoWon't you need a way to figure out the sign of the vector, though? Am I wrong in that this method only output the norms?
- banachtarski 7y agoYes but whether such conditioning is important will depend on your domain. In many cases, just knowing the axis and two sample points is enough to fully determine the orientation.
- mxcrossb 7y agoCan you go into more detail? I don’t see how that would work. In this algorithm, you need the eigenvalues of all of the minors. So if I’m adding rows/columns to the matrix, yes I have one of the minors from my last solution, but I have to compute N more. And I also need the eigenvalues of the new full matrix. I think this result has some intriguing opportunities, and have been toying with them since yesterday’s post, but I don’t see something so straightforward.
- aj7 7y agoSince I’ve always been terrible at abstract linear algebra, I would request that someone do a simple problem, say finding the eigenvalues and eigenvectors of a 3 x 3 Hermitian matrix with three distinct eigenvalues. I’d also be curious under what circumstances the above method leads to a computational advantage.
- aj7 7y agoAnybody?
- rossmohax 7y agoWill it boost FPS in 3d games?
- pgt 7y agoI have a sneaky suspicion this finding can used to speed up expert system matching (e.g. Rete networks), which would affect rendering and compilation times for lots of software.
- acd 7y agoWhen I see people who look similar I think of them as having similar Eigenfaces!
- jjgreen 7y agoEigenfaces are a thing! https://en.wikipedia.org/wiki/Eigenface https://en.wikipedia.org/wiki/Eigenface
- j1vms 7y agoSomewhat related to some of the questions that have been raised here: - for the set of matrices that possess them ("transformation matrices that only perform stretch/contract"), eigenvectors (with their associated eigenvalues) play a role quite analogous to the role primes play in the integer set. They provide a unique, identifying "spectrum" for said matrix. This is made explicit by eigendecomposition (spectral decomposition). - with extension via singular value decomposition (SVD method) to any square matrix (e.g. "transformation matrices that might also shear, rotate"), certain operations such as exponentiation of the square matrix can performed very quickly once eigenvectors/eigenvalues have been obtained via the SVD method.
- creative-coder 7y agoWow... your comment is infact more useful pearl of wisdom for getting the intuition of Eigen-values/vectors. Can you suggest some book/reference that best conveys this intuition ? Somehow, Gilbert Strang's Linear Algebra does put me to sleep :/
- etatoby 7y agoI recommend the Linear Algebra series of YouTube videos by 3blue1brown.
- nimish 7y agoLinear algebra done right by Sheldon axler Linear algebra that focuses on matrices of numbers is missing the point and a proper numerical linear algebra class will teach them better anyway
- btilly 7y agoI would recommend Down with Determinants! instead. Same author, and was the inspiration for that textbook. https://www.maa.org/sites/default/files/pdf/awards/Axler-Ford-1996.pdf https://www.maa.org/sites/default/files/pdf/awards/Axler-For... As a graduate student at the time, it opened my eyes about a whole bunch of theorems that I thought I already knew. And it is a lot more approachable than a textbook. It won an award for the best piece of expository writing on mathematics in 1996. It was a deserved win. :-)
- ur-whale 7y agoBefore you excitedly jump to coding a new algorithm to produce eigenvectors from eigenvalues, it's worth noting that: . This only applies to hermitian matrices and is therefore of limited scope. . The identity only produces the magnitudes of the coefficients of the eigenvectors. The signs of the coefficient sare still unknown. . Even if knowing only the magnitude of the coefficients is somehow useful, this is still very expensive to compute - just like Cramer rule which is of great theoretical interest, but has very little practical use when actually computing a matrix inverse or a determinant. None of the above diminishes the importance of the result, btw.
- gajomi 7y agoCame here to pretty much write these same notes. The fact that you need to know the spectrum of all the minors really limits the usefulness for solving general eigen* problems. But I can imagine that this could make a great starting point for all kinds of approximate calculations/asymptotic analyses. Regarding the first caveat that you bring up though, whereas the problem statement says you need a Hermitian matrix I think the results should generalize to non-hermitian matrices. In particular take a look at the third proof of the theorem at the end of the article. The only assumption required here is that the eigenvalues are simple (which does not preclude them being complex/coming in complex pairs). Protip: I had to read the second step in the proof a few times before I could see what was going on. Explicitly what you do here is to (i) multiple from the right by the elementary unit vector e_j (ii) set up the matrix vector inverse using Cramer's rule (iii) notice that this matrix can be permuted to have a block diagonal element equal to the minor M_i with the other block diagonal entry equal to 1 and the corresponding column equal filled with zeros (iv) Write the determinant in the block form, which simplifies to the expression involving M_i by itself then finally (v) multiply by e_j from the left to get scalar equation.
- MattSayar 7y agoDespite the limited scope, this information is "out there" now, and could potentially be crucial for some future "useful" application. Not that I think you're trying to downplay it or anything, I'm just seeing this as another example of how the internet is still a good thing.
- math_and_stuff 7y agoThe entries of a matrix are eigenvalues of 1x1 minors, and clearly eigenvectors are a function of the entries. That the result is a function of (n-1) minors is first-order information and would help clarify the default assumption that the result computes eigenvectors from the full Hermitian matrix's eigenvalues.
- angel_j 7y agoDoes this mean you can test large neural networks with smaller ones?
- c1b 7y agoAre your neural networks parameterized as Hermitian matrices? (No.)
- kgwgk 7y ago"The real question is whether the discovery is important or interesting. And whether it is surprising. [...] OK, why does Wolchover claim that it is interesting or surprising? Because they gave that formula to Terence Tao, a Fields Medal winner, for him to say something and he didn't know the formula and even expressed doubts. [...] So that's how it works. If you want to promote a new chocolate candybar, send a formula – or the nutritional values from the package – to someone like Terence Tao, to a darling of the media. The popular science writers will do the rest." https://motls.blogspot.com/2019/11/hype-about-formula-for-eigenstates.html https://motls.blogspot.com/2019/11/hype-about-formula-for-ei...
- jl2718 7y agoI believe this is equivalent to the solution given by Gene Golub in 1973 for the constraint vector required to produce a specific set of stationary values in a linearly-constrained quadratic program where the constraint vector is unitary. https://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=9B05FFCB5C07BC4A1A2A00C7FF2C2C68?doi=10.1.1.454.9868&rep=rep1&type=pdf https://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=9B... (eq. 3.6) I haven't looked at Tao's proofs to see if this is covered, nor do I have a clear formulation of the equivalence, but it certainly is striking.
- Noxmiles 7y agoReading german words, still don't understand anything...
- pgt 7y agoThis can probably be used to speed up Transfer Learning.