11 ms·

Eigenvectors and Eigenvalues (2015)

369 points by vimalvnair 8y ago

danlugo92 8y ago
I highly recommend 1Blue1Brown's Essence of Linear Algebra series[0] to highly grasp and comprehend linear algebra. [0] https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...
- olskool 8y ago
  This is truly a favorite of mine.
xevb3k 8y ago
Whenever this kind of stuff comes up I feel like a bit of a fraud... I’ve written a bunch of scientific data analysis code. I have a science PhD. Written large image analysis pipelines that worked as well as the state of the art... been published etc. For the most part I’ve found basic math and heuristics to be good enough. Every so often I go relearn calculus. But honestly, none of this stuff ever seems to come in handy. Maybe it’s because most of what I encounter is novel datasets where there’s no established method? I reasonably regularly pick up new discrete methods, but the numerical stuff never seems super useful... I don’t know, just a confession I guess... it never comes up on interviews either for what it’s worth.
- rxhernandez 8y ago
  Just my anecdotal experience, I've seen it come up in interviews for an algorithms group at a medium sized biomed company.
- tzahola 8y ago
  What kind of novel dataset are we talking about? Just from the top of my head, you could have encountered eigenvectors/eigenvalues: - if you ever used spectral graph algorithms - if you ever done dimensionality reduction via principal component analysis - if you ever calculated the steady state distribution of a Markov chain
  - xevb3k 8y ago
    Image analysis and signal corrections on time series data. Extracting features from microscope images for example, correcting for signal convolutions.
- jdonaldson 8y ago
  Eigenvectors and Eigenvalues show up everywhere, although sometimes it's in the form of an iterative estimate (PageRank is basically the power method estimation of the first eigenvector of a connected graph of web pages). They're in the same class as logarithms and Fourier transforms IMHO. You won't need to calculate them by hand, but you should know what they do and why they're important.
  - xevb3k 8y ago
    Interesting, perhaps that’s it... I’ve used FFTs a few times, but even FFTs have never been make or break in terms of getting a pipeline or analysis working well.
    - cowboysauce 8y ago
      The Fourier transform is actually linked with linear algebra. You can think of it as taking a vector in an infinite dimensional Hilbert space (your signal) and decomposing it into its components (the amplitudes of the frequencies).
      - stephencanon 8y ago
        The Fourier transform as used in practice in signal processing is ~always discrete, hence finite-dimensional. It's literally "just" a rotation with an especially nice decomposition that allows efficient multiplication.
        danharaj 8y ago
        A particular fft in practice may be discrete but the relation between fft's of different resolutions of the "same" signal hints at the infinite structure which bundles up all the finite subspaces into one conceptual object.
  - WhompingWindows 8y ago
    Why should you know what they do and why they're important? How does that practically change my R code?
- electricslpnsld 8y ago
  It comes up all the time when trying to build second order optimization methods. With the eigensystem of your objective in hand you have a complete understanding of the (non-) convexity of your energy landscape, which is useful to ensure you always have good search directions, etc.
- aje403 8y ago
  This is frightening but believable. I've worked with a few "quants" who stared at me doe eyed explaining eigen* and basic calculus concepts to them in the context of why their calculations don't add up. You mention you've used fourier transforms before - if you don't understand an eigenbasis then you don't have a fundamental understanding the math you're deploying.
  - xevb3k 8y ago
    In the context I used it, FFT wasn’t really of fundamental importance. Using FFT (and in particular FFTW) gave a performance improvement (execution speed), but no real advantage in terms of accuracy over an alternative naive method... So... yes I guess I’ve just not seen anywhere in my work where this stuff has proved useful...
    - aje403 8y ago
      It's pretty much ubiquitous in any quantitative field... would not even know where to start
  - ajross 8y ago
    > You mention you've used fourier transforms before - if you don't understand an eigenbasis then you don't have a fundamental understanding the math you're deploying. That's a bit uncharitable. A fourier decomposition can absolutely be understood as an explicit bag of calculus tricks, with no loss of precision or generality. And an awful lot can be done with just those tools -- you don't need to explain JPEG compression or VLBI astronomy in terms of eigenvectors, for example. Obviously (heh, "obviously") it's true that the space of decomposed functions form an orthogonal basis, so technically we're "really" operating in a linear space and that has expressive power too. But there are lots of ways of looking at problems. To wit, you're not wrong. You're just... Well, you know.
    - aje403 8y ago
      Dude your post is orthogonal
- soVeryTired 8y ago
  For a large fraction of probability theory, you only need two main facts from linear algebra. First, linear transforms map spheres to ellipsoids. The axes of the ellipsoid are the eigenvectors. Second, linear transforms map (hyper) cubes to parallelpipeds. If you start with a unit cube, the volume of the parallelpiped is the determinant of the transform. That more or less covers covariances, PCA, and change of variables. Whenever I try to understand or re-derive a fact in probability, I almost always end up back at one or the other fact. They're also useful in multivariate calculus, which is really just stitched-together linear algebra.
  - sakuronto 8y ago
    Not a very useful addition but hypercube is to cube as parallelotope is to parallelepiped.
  - zawerf 8y ago
    I use the 2nd point a lot for debugging 3d transforms. To expand upon it, for example in three dimensions the three axes are: (1, 0, 0) (0, 1, 0) (0, 0, 1) To find out where those axes are after a 3x3 matrix transform, you just read off the first, second, and third columns of the matrix respectively. Then you can mentally visualize another unit cube in the new coordinate system using those three vectors as the edges of the cube. Really basic change-of-basis stuff but academic lectures don't emphasize how useful it is to be able to look at a matrix of numbers immediately know what it does.
    - aidos 8y ago
      This concept totally changed my intuitive understanding of matrices. Beautifully illustrated in the below 3blue1brown video. https://youtu.be/kYB8IZa5AuE?t=3m15s https://youtu.be/kYB8IZa5AuE?t=3m15s
  - qmalzp 8y ago
    I think the first point is only true for symmetric matrices (which includes those that show up in multivariable calc). In general, the eigenvectors need not be orthogonal.
- Bukhmanizer 8y ago
  It's one of those things that you don't notice when it's missing, but probably would help a bit if you knew it. That being said, I have to deal with linear algebra every day, and aside from proofs (which obviously they help with), there have been maybe a handful of times that having a deep knowledge of eigenvectors and eigenvalues has helped significantly. Once or twice though, I've got massive speedups (>500x) just by knowing how to do the same thing in a more efficient way. My feeling is having a basic knowledge of testing/caching/memory management is way more useful when you're doing large image analysis.
  - xevb3k 8y ago
    Interesting, well I’ll try to keep reviewing this stuff and hoping I find an application. I really would like to find an application in my work, because without that I find new techniques don’t really stick and after a few months I forget them...
    - Bukhmanizer 8y ago
      I'm the exact same way, my job is really heavy in linear algebra, so it sticks more easily for me. Usually I go through the code and ask "what am I trying to do here" and "can I do this a better way". A lot of the aforementioned speedups have come because the previous developer was obviously trying to do something, like create a linear projector, but were following some sort of math formula, so made a bunch of extraneous matrices that were huge. It's a simple fix, but adds up when you're dealing with massive datasets.
    - gh02t 8y ago
      It depends on the field you're in. For example, if you're in an area that heavily uses differential equations (many engineering disciplines) then you're probably gonna be using eigenvectors a lot, as they are important for solving a lot of problems. Other areas may not need them at all. It also depends on your depth in the field. A rank and file engineer may not need to know anything about them - they underpin a lot of numerical methods, but get hidden away in software packages. Someone developing those software packages likely will, though. Techniques based on eigenvectors and eigenvalues are extremely important in my field (nuclear engineering... you've probably heard the term "critical", that refers to an eigenvalue), but I know someone who is an excellent civil engineer and knows next to nothing about them (or linear algebra in general) because they aren't that important for what he works on. Forgetting stuff you don't use is pretty normal, the important thing is to be able to recognize when a technique you don't remember the details of might be applicable, and to know where to look to refresh your memory.
- wglb 8y ago
  It is certainly frequent in engineering. If you need to analyze the stability of an electrical grid, there isn't much alternative. A fun book on this is https://openlibrary.org/books/OL2398351M/The_algebraic_eigenvalue_problem https://openlibrary.org/books/OL2398351M/The_algebraic_eigen...
steamer25 8y ago
3Blue1Brown has a good series on YouTube for building intuition in linear algebra: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x... In one of the last videos in the (relatively short) series, he discusses eigen-*: ~'eigen-stuffs are straight-forward but only make sense if you have a solid visual understanding of the pre-requisites (linear transformations, determinants, linear systems of equations, change of basis, etc.). Confusion about eigen-stuffs usually has more to do with a shaky foundation than the eigen-things themselves' https://youtu.be/PFDu9oVAE-g https://youtu.be/PFDu9oVAE-g All of the videos in the series, including this later one on eigen-things, focus on animations to show what the number crunching is doing to the coordinate system.
- AceJohnny2 8y ago
  3Blue1Brown (Grant Sanderson) is really, really good. I follow a number of education channels on YouTube, and Grant blows them all out of the water for the kind of insights, new perspectives, and inspiration he provides. His animations are fantastically put together to clearly and unobtrusively illustrate the point he's making. I also really like his voice, soothing, clear and with enough intonation to avoid boredom, and perfect pace. I wish I had his linear algebra series back in college, I suspect I would have done much better. He's the creator I support the most on Patreon: https://www.patreon.com/3blue1brown https://www.patreon.com/3blue1brown
  - bashcorrupt 8y ago
    >I follow a number of education channels on YouTube Any recommendations? I would love to look into channels that you think are like 3Blue1Brown but in other subjects (natural sciences, history, art etc.).
    - AceJohnny2 8y ago
      Edit: I realize I overlooked the "like 3Blue1Brown" prereq, instead sharing a list of the educational/interesting channels I find worthwhile. The most like 3Blue1Brown will be the PBS ones (especially SpaceTime), MinutePhysics, and Mathologer, for using diagrams to convey abstract concepts. PBS Space Time and Eons are both awesome: * PBS Space Time, covers cosmology and quantum physics: https://www.youtube.com/channel/UC7_gcs09iThXybpVgjHZ_7g https://www.youtube.com/channel/UC7_gcs09iThXybpVgjHZ_7g * PBS Eons, for geology and paleontology: https://www.youtube.com/channel/UCzR-rom72PHN9Zg7RML9EbA https://www.youtube.com/channel/UCzR-rom72PHN9Zg7RML9EbA * Smarter Every Day, Destin's enthusiasm is contagious: https://www.youtube.com/user/destinws2 https://www.youtube.com/user/destinws2 * Extra Credits various topics (video game design, History, and recently history of Sci-Fi) are great: https://www.youtube.com/user/ExtraCreditz https://www.youtube.com/user/ExtraCreditz * Today I Found Out is just on this side of clickbaity, and is this age's "Ripley's Believe It Or Not", but still interesting and more importantly well researched: https://www.youtube.com/user/TodayIFoundOut https://www.youtube.com/user/TodayIFoundOut * Crash Course, of course: https://www.youtube.com/channel/UCX6b17PVsYBQ0ip5gyeme-Q https://www.youtube.com/channel/UCX6b17PVsYBQ0ip5gyeme-Q * Gaming Historian, for some insight into the making of systems that formed my (and earlier) childhood: https://www.youtube.com/channel/UCnbvPS_rXp4PC21PG2k1UVg https://www.youtube.com/channel/UCnbvPS_rXp4PC21PG2k1UVg * Minute Physics, whose latest few videos made Special Relativity understandable to this peon: https://www.youtube.com/user/minutephysics/videos https://www.youtube.com/user/minutephysics/videos * Practical Engineering, for some insight into civil engineering topics that we take for granted: https://www.youtube.com/channel/UCMOqf8ab-42UUQIdVoKwjlQ https://www.youtube.com/channel/UCMOqf8ab-42UUQIdVoKwjlQ * Real Engineering, for insight into various other mechanical engineering topics: https://www.youtube.com/channel/UCR1IuLEqb6UEA_zQ81kwXfg https://www.youtube.com/channel/UCR1IuLEqb6UEA_zQ81kwXfg * Standup Maths, host Matt Parker was the first to make maths approachable for me again (before 3Blue1Brown took the lead): https://www.youtube.com/channel/UCSju5G2aFaWMqn-_0YBtq5A https://www.youtube.com/channel/UCSju5G2aFaWMqn-_0YBtq5A * Steve Mould, who covers various topics both mathematical and physical. You may have seen that gif of him demonstrating the "levitating" siphoning "pearl necklace" (also a friend of Matt Parker, above): https://www.youtube.com/channel/UCEIwxahdLz7bap-VDs9h35A https://www.youtube.com/channel/UCEIwxahdLz7bap-VDs9h35A * The 8-Bit Guy, for some history of early home computer systems: https://www.youtube.com/channel/UC8uT9cgJorJPWu7ITLGo9Ww https://www.youtube.com/channel/UC8uT9cgJorJPWu7ITLGo9Ww * Numberphile, the second-greatest math channel (after 3Blue1Brown), whose recent video finally made me take the Golden Ratio seriously, rather than an architectural gimmick/conspiracy theory: https://www.youtube.com/user/numberphile https://www.youtube.com/user/numberphile * Mathologer, another good math channel (but I must sheepishly admit I prefer 3Blue1Brown... sensing a pattern here?): https://www.youtube.com/channel/UC1_uAIS3r8Vu6JjXWvastJg https://www.youtube.com/channel/UC1_uAIS3r8Vu6JjXWvastJg * Periodic Videos, for chemistry and physics, often featuring the iconic Dr Martyn Poliakoff: https://www.youtube.com/channel/UCtESv1e7ntJaLJYKIO1FoYw https://www.youtube.com/channel/UCtESv1e7ntJaLJYKIO1FoYw * NileRed, for some homegrown chemistry, I particularly appreciate the candor of the approach and results: https://www.youtube.com/user/TheRedNile https://www.youtube.com/user/TheRedNile Not quite as much "educational", but still very very good: * Every Frame A Painting, now finished, but a great explanation of what makes good cinematography: https://www.youtube.com/channel/UCjFqcJQXGZ6T6sxyFB-5i6A https://www.youtube.com/channel/UCjFqcJQXGZ6T6sxyFB-5i6A * NoClip, long-form documentaries about the making-of video games. Danny O'Dwyer is a treasure: https://www.youtube.com/channel/UC0fDG3byEcMtbOqPMymDNbw https://www.youtube.com/channel/UC0fDG3byEcMtbOqPMymDNbw Apologies for the link spam, this list turned out longer than I expected as I went down my subscriptions, and I've probably missed a few worthy ones! Edit the final: I discovered many of these channels through referrals from others I was watching, including from the twitter feeds of the authors. Turns out the educational landscape on YouTube is a well-connected graph!
haskellandchill 8y ago
The visual explanation movement falls flat for me. It's like trying to understand Monads through blog posts. It's great if you already understand the concept to develop your intuition, or if you've never heard of the concept to pique your interest, but it won't help in the intermediate area where you know what you want to know but don't understand it fully. I need to build proofs through incremental exercises to grasp these concepts.
- whatshisface 8y ago
  As someone who understands eigenfunctions already, I don't understand the pictures either. Here is the best way to think about it: a matrix is a transformation, a composition of rotation, scaling, etc. Eigensets are lines going through the origin that the matrix moves points along. So a rotation would have no eigenvectors because none of the points move in a straight line, while a scaling along the x axis would have an eigenset that was also along the x axis, consisting of the points that were moved straight up or down. To imagine finding the eigenset, just ask, could I draw a line through 0,0 such that any point I put on it would stay on it after the matrix acted?
  - electricslpnsld 8y ago
    > So a rotation would have no eigenvectors Rotations have eigenvectors: a 2D rotation has two complex eigenvectors, a 3D rotation has one real and two complex eigenvectors, ...
    - whatshisface 8y ago
      That's a fair and true catch, but I can cover myself by pointing out that the article was only talking about matrices in R^(m x n). ;)
    - sakuronto 8y ago
      When your field of interest is the reals, those complex eigenvectors don't matter.
  - 8y ago
WhompingWindows 8y ago
Am I the only one with [Math Processing Error] all over this source? Ctrl+F gives me 57 instances of that string
- abiox 8y ago
  i didn't see anything like that. possibly a mathjax.org hiccup?
- akvadrako 8y ago
  I see that on some pages. Try reloading.
KasianFranks 8y ago
There's nothing to see here.
- abiox 8y ago
  what do you mean? this seems like a strange statement, given the context.
lewis500 8y ago
Interesting to see this back on the front page after three years. Still remember us sitting in our living room drawing this on paper and arguing about the right approaches. Maybe one day vicapow and I will make a triumphant return to the explorables space, but life has a way of getting in the way as you get older.
- dang 8y ago
  That was https://news.ycombinator.com/item?id=8918259 https://news.ycombinator.com/item?id=8918259.
lr4444lr 8y ago
Great interactive demos.
sannee 8y ago
Eigen{vectors,values} seemed like this totally arbitrary concept when I first learned about them. Later it turned out that they are actually really awesome and pop up all the time. Multivariable function extrema? Just look at the eigenvalues of the hessian. Jacobi method convergence? Eigenvalues of the update matrix. RNN gradient explosion? Of course, eigenvalues.
deepdiving12 8y ago
Beautiful demos/explanations. Would've been really handy back in school.
toppy 8y ago
Classic paper on Google's PageRank: "The $25,000,000,000 eigenvector" https://www.rose-hulman.edu/~bryan/googleFinalVersionFixed.pdf https://www.rose-hulman.edu/~bryan/googleFinalVersionFixed.p...
asafira 8y ago
Quantum mechanics should be listed as another reason to learn about eigenvectors and eigenvalues! =)

369 points by vimalvnair 8y ago

I highly recommend 1Blue1Brown's Essence of Linear Algebra series[0] to highly grasp and comprehend linear algebra. [0] https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...

olskool 8y ago
This is truly a favorite of mine.

xevb3k 8y ago

Whenever this kind of stuff comes up I feel like a bit of a fraud... I’ve written a bunch of scientific data analysis code. I have a science PhD. Written large image analysis pipelines that worked as well as the state of the art... been published etc. For the most part I’ve found basic math and heuristics to be good enough. Every so often I go relearn calculus. But honestly, none of this stuff ever seems to come in handy. Maybe it’s because most of what I encounter is novel datasets where there’s no established method? I reasonably regularly pick up new discrete methods, but the numerical stuff never seems super useful... I don’t know, just a confession I guess... it never comes up on interviews either for what it’s worth.

rxhernandez 8y ago
Just my anecdotal experience, I've seen it come up in interviews for an algorithms group at a medium sized biomed company.
tzahola 8y ago
What kind of novel dataset are we talking about? Just from the top of my head, you could have encountered eigenvectors/eigenvalues: - if you ever used spectral graph algorithms - if you ever done dimensionality reduction via principal component analysis - if you ever calculated the steady state distribution of a Markov chain
- xevb3k 8y ago
  Image analysis and signal corrections on time series data. Extracting features from microscope images for example, correcting for signal convolutions.
jdonaldson 8y ago
Eigenvectors and Eigenvalues show up everywhere, although sometimes it's in the form of an iterative estimate (PageRank is basically the power method estimation of the first eigenvector of a connected graph of web pages). They're in the same class as logarithms and Fourier transforms IMHO. You won't need to calculate them by hand, but you should know what they do and why they're important.
- xevb3k 8y ago
  Interesting, perhaps that’s it... I’ve used FFTs a few times, but even FFTs have never been make or break in terms of getting a pipeline or analysis working well.
  - cowboysauce 8y ago
    The Fourier transform is actually linked with linear algebra. You can think of it as taking a vector in an infinite dimensional Hilbert space (your signal) and decomposing it into its components (the amplitudes of the frequencies).
    - stephencanon 8y ago
      The Fourier transform as used in practice in signal processing is ~always discrete, hence finite-dimensional. It's literally "just" a rotation with an especially nice decomposition that allows efficient multiplication.
      - danharaj 8y ago
        A particular fft in practice may be discrete but the relation between fft's of different resolutions of the "same" signal hints at the infinite structure which bundles up all the finite subspaces into one conceptual object.
- WhompingWindows 8y ago
  Why should you know what they do and why they're important? How does that practically change my R code?
electricslpnsld 8y ago
It comes up all the time when trying to build second order optimization methods. With the eigensystem of your objective in hand you have a complete understanding of the (non-) convexity of your energy landscape, which is useful to ensure you always have good search directions, etc.
aje403 8y ago
This is frightening but believable. I've worked with a few "quants" who stared at me doe eyed explaining eigen* and basic calculus concepts to them in the context of why their calculations don't add up. You mention you've used fourier transforms before - if you don't understand an eigenbasis then you don't have a fundamental understanding the math you're deploying.
- xevb3k 8y ago
  In the context I used it, FFT wasn’t really of fundamental importance. Using FFT (and in particular FFTW) gave a performance improvement (execution speed), but no real advantage in terms of accuracy over an alternative naive method... So... yes I guess I’ve just not seen anywhere in my work where this stuff has proved useful...
  - aje403 8y ago
    It's pretty much ubiquitous in any quantitative field... would not even know where to start
- ajross 8y ago
  > You mention you've used fourier transforms before - if you don't understand an eigenbasis then you don't have a fundamental understanding the math you're deploying. That's a bit uncharitable. A fourier decomposition can absolutely be understood as an explicit bag of calculus tricks, with no loss of precision or generality. And an awful lot can be done with just those tools -- you don't need to explain JPEG compression or VLBI astronomy in terms of eigenvectors, for example. Obviously (heh, "obviously") it's true that the space of decomposed functions form an orthogonal basis, so technically we're "really" operating in a linear space and that has expressive power too. But there are lots of ways of looking at problems. To wit, you're not wrong. You're just... Well, you know.
  - aje403 8y ago
    Dude your post is orthogonal
soVeryTired 8y ago
For a large fraction of probability theory, you only need two main facts from linear algebra. First, linear transforms map spheres to ellipsoids. The axes of the ellipsoid are the eigenvectors. Second, linear transforms map (hyper) cubes to parallelpipeds. If you start with a unit cube, the volume of the parallelpiped is the determinant of the transform. That more or less covers covariances, PCA, and change of variables. Whenever I try to understand or re-derive a fact in probability, I almost always end up back at one or the other fact. They're also useful in multivariate calculus, which is really just stitched-together linear algebra.
- sakuronto 8y ago
  Not a very useful addition but hypercube is to cube as parallelotope is to parallelepiped.
- zawerf 8y ago
  I use the 2nd point a lot for debugging 3d transforms. To expand upon it, for example in three dimensions the three axes are: (1, 0, 0) (0, 1, 0) (0, 0, 1) To find out where those axes are after a 3x3 matrix transform, you just read off the first, second, and third columns of the matrix respectively. Then you can mentally visualize another unit cube in the new coordinate system using those three vectors as the edges of the cube. Really basic change-of-basis stuff but academic lectures don't emphasize how useful it is to be able to look at a matrix of numbers immediately know what it does.
  - aidos 8y ago
    This concept totally changed my intuitive understanding of matrices. Beautifully illustrated in the below 3blue1brown video. https://youtu.be/kYB8IZa5AuE?t=3m15s https://youtu.be/kYB8IZa5AuE?t=3m15s
- qmalzp 8y ago
  I think the first point is only true for symmetric matrices (which includes those that show up in multivariable calc). In general, the eigenvectors need not be orthogonal.
Bukhmanizer 8y ago
It's one of those things that you don't notice when it's missing, but probably would help a bit if you knew it. That being said, I have to deal with linear algebra every day, and aside from proofs (which obviously they help with), there have been maybe a handful of times that having a deep knowledge of eigenvectors and eigenvalues has helped significantly. Once or twice though, I've got massive speedups (>500x) just by knowing how to do the same thing in a more efficient way. My feeling is having a basic knowledge of testing/caching/memory management is way more useful when you're doing large image analysis.
- xevb3k 8y ago
  Interesting, well I’ll try to keep reviewing this stuff and hoping I find an application. I really would like to find an application in my work, because without that I find new techniques don’t really stick and after a few months I forget them...
  - Bukhmanizer 8y ago
    I'm the exact same way, my job is really heavy in linear algebra, so it sticks more easily for me. Usually I go through the code and ask "what am I trying to do here" and "can I do this a better way". A lot of the aforementioned speedups have come because the previous developer was obviously trying to do something, like create a linear projector, but were following some sort of math formula, so made a bunch of extraneous matrices that were huge. It's a simple fix, but adds up when you're dealing with massive datasets.
  - gh02t 8y ago
    It depends on the field you're in. For example, if you're in an area that heavily uses differential equations (many engineering disciplines) then you're probably gonna be using eigenvectors a lot, as they are important for solving a lot of problems. Other areas may not need them at all. It also depends on your depth in the field. A rank and file engineer may not need to know anything about them - they underpin a lot of numerical methods, but get hidden away in software packages. Someone developing those software packages likely will, though. Techniques based on eigenvectors and eigenvalues are extremely important in my field (nuclear engineering... you've probably heard the term "critical", that refers to an eigenvalue), but I know someone who is an excellent civil engineer and knows next to nothing about them (or linear algebra in general) because they aren't that important for what he works on. Forgetting stuff you don't use is pretty normal, the important thing is to be able to recognize when a technique you don't remember the details of might be applicable, and to know where to look to refresh your memory.
wglb 8y ago
It is certainly frequent in engineering. If you need to analyze the stability of an electrical grid, there isn't much alternative. A fun book on this is https://openlibrary.org/books/OL2398351M/The_algebraic_eigenvalue_problem https://openlibrary.org/books/OL2398351M/The_algebraic_eigen...

steamer25 8y ago

3Blue1Brown has a good series on YouTube for building intuition in linear algebra: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x... In one of the last videos in the (relatively short) series, he discusses eigen-*: ~'eigen-stuffs are straight-forward but only make sense if you have a solid visual understanding of the pre-requisites (linear transformations, determinants, linear systems of equations, change of basis, etc.). Confusion about eigen-stuffs usually has more to do with a shaky foundation than the eigen-things themselves' https://youtu.be/PFDu9oVAE-g https://youtu.be/PFDu9oVAE-g All of the videos in the series, including this later one on eigen-things, focus on animations to show what the number crunching is doing to the coordinate system.

AceJohnny2 8y ago
3Blue1Brown (Grant Sanderson) is really, really good. I follow a number of education channels on YouTube, and Grant blows them all out of the water for the kind of insights, new perspectives, and inspiration he provides. His animations are fantastically put together to clearly and unobtrusively illustrate the point he's making. I also really like his voice, soothing, clear and with enough intonation to avoid boredom, and perfect pace. I wish I had his linear algebra series back in college, I suspect I would have done much better. He's the creator I support the most on Patreon: https://www.patreon.com/3blue1brown https://www.patreon.com/3blue1brown
- bashcorrupt 8y ago
  >I follow a number of education channels on YouTube Any recommendations? I would love to look into channels that you think are like 3Blue1Brown but in other subjects (natural sciences, history, art etc.).
  - AceJohnny2 8y ago
    Edit: I realize I overlooked the "like 3Blue1Brown" prereq, instead sharing a list of the educational/interesting channels I find worthwhile. The most like 3Blue1Brown will be the PBS ones (especially SpaceTime), MinutePhysics, and Mathologer, for using diagrams to convey abstract concepts. PBS Space Time and Eons are both awesome: * PBS Space Time, covers cosmology and quantum physics: https://www.youtube.com/channel/UC7_gcs09iThXybpVgjHZ_7g https://www.youtube.com/channel/UC7_gcs09iThXybpVgjHZ_7g * PBS Eons, for geology and paleontology: https://www.youtube.com/channel/UCzR-rom72PHN9Zg7RML9EbA https://www.youtube.com/channel/UCzR-rom72PHN9Zg7RML9EbA * Smarter Every Day, Destin's enthusiasm is contagious: https://www.youtube.com/user/destinws2 https://www.youtube.com/user/destinws2 * Extra Credits various topics (video game design, History, and recently history of Sci-Fi) are great: https://www.youtube.com/user/ExtraCreditz https://www.youtube.com/user/ExtraCreditz * Today I Found Out is just on this side of clickbaity, and is this age's "Ripley's Believe It Or Not", but still interesting and more importantly well researched: https://www.youtube.com/user/TodayIFoundOut https://www.youtube.com/user/TodayIFoundOut * Crash Course, of course: https://www.youtube.com/channel/UCX6b17PVsYBQ0ip5gyeme-Q https://www.youtube.com/channel/UCX6b17PVsYBQ0ip5gyeme-Q * Gaming Historian, for some insight into the making of systems that formed my (and earlier) childhood: https://www.youtube.com/channel/UCnbvPS_rXp4PC21PG2k1UVg https://www.youtube.com/channel/UCnbvPS_rXp4PC21PG2k1UVg * Minute Physics, whose latest few videos made Special Relativity understandable to this peon: https://www.youtube.com/user/minutephysics/videos https://www.youtube.com/user/minutephysics/videos * Practical Engineering, for some insight into civil engineering topics that we take for granted: https://www.youtube.com/channel/UCMOqf8ab-42UUQIdVoKwjlQ https://www.youtube.com/channel/UCMOqf8ab-42UUQIdVoKwjlQ * Real Engineering, for insight into various other mechanical engineering topics: https://www.youtube.com/channel/UCR1IuLEqb6UEA_zQ81kwXfg https://www.youtube.com/channel/UCR1IuLEqb6UEA_zQ81kwXfg * Standup Maths, host Matt Parker was the first to make maths approachable for me again (before 3Blue1Brown took the lead): https://www.youtube.com/channel/UCSju5G2aFaWMqn-_0YBtq5A https://www.youtube.com/channel/UCSju5G2aFaWMqn-_0YBtq5A * Steve Mould, who covers various topics both mathematical and physical. You may have seen that gif of him demonstrating the "levitating" siphoning "pearl necklace" (also a friend of Matt Parker, above): https://www.youtube.com/channel/UCEIwxahdLz7bap-VDs9h35A https://www.youtube.com/channel/UCEIwxahdLz7bap-VDs9h35A * The 8-Bit Guy, for some history of early home computer systems: https://www.youtube.com/channel/UC8uT9cgJorJPWu7ITLGo9Ww https://www.youtube.com/channel/UC8uT9cgJorJPWu7ITLGo9Ww * Numberphile, the second-greatest math channel (after 3Blue1Brown), whose recent video finally made me take the Golden Ratio seriously, rather than an architectural gimmick/conspiracy theory: https://www.youtube.com/user/numberphile https://www.youtube.com/user/numberphile * Mathologer, another good math channel (but I must sheepishly admit I prefer 3Blue1Brown... sensing a pattern here?): https://www.youtube.com/channel/UC1_uAIS3r8Vu6JjXWvastJg https://www.youtube.com/channel/UC1_uAIS3r8Vu6JjXWvastJg * Periodic Videos, for chemistry and physics, often featuring the iconic Dr Martyn Poliakoff: https://www.youtube.com/channel/UCtESv1e7ntJaLJYKIO1FoYw https://www.youtube.com/channel/UCtESv1e7ntJaLJYKIO1FoYw * NileRed, for some homegrown chemistry, I particularly appreciate the candor of the approach and results: https://www.youtube.com/user/TheRedNile https://www.youtube.com/user/TheRedNile Not quite as much "educational", but still very very good: * Every Frame A Painting, now finished, but a great explanation of what makes good cinematography: https://www.youtube.com/channel/UCjFqcJQXGZ6T6sxyFB-5i6A https://www.youtube.com/channel/UCjFqcJQXGZ6T6sxyFB-5i6A * NoClip, long-form documentaries about the making-of video games. Danny O'Dwyer is a treasure: https://www.youtube.com/channel/UC0fDG3byEcMtbOqPMymDNbw https://www.youtube.com/channel/UC0fDG3byEcMtbOqPMymDNbw Apologies for the link spam, this list turned out longer than I expected as I went down my subscriptions, and I've probably missed a few worthy ones! Edit the final: I discovered many of these channels through referrals from others I was watching, including from the twitter feeds of the authors. Turns out the educational landscape on YouTube is a well-connected graph!

haskellandchill 8y ago

The visual explanation movement falls flat for me. It's like trying to understand Monads through blog posts. It's great if you already understand the concept to develop your intuition, or if you've never heard of the concept to pique your interest, but it won't help in the intermediate area where you know what you want to know but don't understand it fully. I need to build proofs through incremental exercises to grasp these concepts.

whatshisface 8y ago
As someone who understands eigenfunctions already, I don't understand the pictures either. Here is the best way to think about it: a matrix is a transformation, a composition of rotation, scaling, etc. Eigensets are lines going through the origin that the matrix moves points along. So a rotation would have no eigenvectors because none of the points move in a straight line, while a scaling along the x axis would have an eigenset that was also along the x axis, consisting of the points that were moved straight up or down. To imagine finding the eigenset, just ask, could I draw a line through 0,0 such that any point I put on it would stay on it after the matrix acted?
- electricslpnsld 8y ago
  > So a rotation would have no eigenvectors Rotations have eigenvectors: a 2D rotation has two complex eigenvectors, a 3D rotation has one real and two complex eigenvectors, ...
  - whatshisface 8y ago
    That's a fair and true catch, but I can cover myself by pointing out that the article was only talking about matrices in R^(m x n). ;)
  - sakuronto 8y ago
    When your field of interest is the reals, those complex eigenvectors don't matter.
- 8y ago

WhompingWindows 8y ago

Am I the only one with [Math Processing Error] all over this source? Ctrl+F gives me 57 instances of that string

abiox 8y ago
i didn't see anything like that. possibly a mathjax.org hiccup?
akvadrako 8y ago
I see that on some pages. Try reloading.

KasianFranks 8y ago

There's nothing to see here.

abiox 8y ago
what do you mean? this seems like a strange statement, given the context.

lewis500 8y ago

Interesting to see this back on the front page after three years. Still remember us sitting in our living room drawing this on paper and arguing about the right approaches. Maybe one day vicapow and I will make a triumphant return to the explorables space, but life has a way of getting in the way as you get older.

dang 8y ago
That was https://news.ycombinator.com/item?id=8918259 https://news.ycombinator.com/item?id=8918259.

lr4444lr 8y ago

Great interactive demos.

sannee 8y ago

Eigen{vectors,values} seemed like this totally arbitrary concept when I first learned about them. Later it turned out that they are actually really awesome and pop up all the time. Multivariable function extrema? Just look at the eigenvalues of the hessian. Jacobi method convergence? Eigenvalues of the update matrix. RNN gradient explosion? Of course, eigenvalues.

deepdiving12 8y ago

Beautiful demos/explanations. Would've been really handy back in school.

toppy 8y ago

Classic paper on Google's PageRank: "The $25,000,000,000 eigenvector" https://www.rose-hulman.edu/~bryan/googleFinalVersionFixed.pdf https://www.rose-hulman.edu/~bryan/googleFinalVersionFixed.p...

asafira 8y ago

Quantum mechanics should be listed as another reason to learn about eigenvectors and eigenvalues! =)