11 ms·

This is a cool example of what Bret Victor calls an "Explorable Explanation" [0]. That said, I feel that it's more important to understand how and why matrix mu

This is a cool example of what Bret Victor calls an "Explorable Explanation" [0]. That said, I feel that it's more important to understand how and why matrix multiplication corresponds to a composition of linear transformations than learning the actual mechanics of doing the computation. You can get good at matrix multiplication without knowing what is going on. I view that as a less valuable activity than learning about linear transformations (vector-space structure preserving mappings) between finite dimensional vector spaces. [0] http://worrydream.com/ExplorableExplanations/ http://worrydream.com/ExplorableExplanations/

bandrami 10y ago
Bingo. We did matrices in high school and even through a first linear algebra course in undergrad I got more or less competent at the mechanics of matrix multiplication without ever having any sense of why anyone would want to do this or why these rules, as opposed to some other set of rules that were equally formally valid, were of any interest. I could even tell you what the eigenvectors of a matrix were but had no sense of what that actually meant in terms of a linear transformation that matrix represented. Linear algebra is probably the area of math that is both universally important and completely skimped on by our education system.
- t3nary 10y ago
  I feel very similar. Do you have any recommendations for resources to learn about why matrix multiplication is defined that way?
  - darrickw 10y ago
    Check out this series of videos "The Essence of Linear Algebra"[1] for a really powerful visual and intuitive explanation. It starts with vectors and builds to matrix multiplication and further to several other topics. [1]: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...
  - bandrami 10y ago
    "All the mathematics you missed but need to know for graduate school"[1] helped me a lot (and, in fact, I had and did). Once I had finished that, Cullen's "Matrices and linear transformations"[2] was really helpful too. But I wouldn't do Cullen if you're still, as I was, floundering with the concepts of why you're doing this in the first place. It's great once you have those concepts down. [1]: https://www.amazon.com/All-Mathematics-You-Missed-Graduate/dp/0521797071 https://www.amazon.com/All-Mathematics-You-Missed-Graduate/d... [2]: http://store.doverpublications.com/0486663280.html http://store.doverpublications.com/0486663280.html
    - fsloth 10y ago
      Garrity's book looks exactly what I've been looking for for brushing up on basic math skills after a decade of fairly low-brow work after my MSc. Thanks for the reference!
      - bandrami 10y ago
        Hope it helps! It definitely got me through the first year of grad school...
  - cousin_it 10y ago
    Let's say you have a system with N possible states, evolving in discrete steps. At each step, the system has some probability of switching from any state to any other state. That gives you an NxN matrix of switching probabilities. For example, if the system always stays in the same state as it started, the switching probabilities are an identity matrix (1 on the diagonal and 0 everywhere else). Now let's see what happens after two steps. If the system started out in state i, what's the probability that after two steps it will end up in state k? Well, it's the sum over all possible paths. In other words, the sum of probabilities of i->j->k for all possible j. In other words, the sum of p_{ij} times p_{jk} for j from 1 to N. But that's exactly the definition of multiplying a matrix by itself. Now it should be easy to understand that whenever you have matrices that represent transformations of some object, composing transformations will correspond to multiplying matrices.
  - ww2 10y ago
    If you compose two linear transformations into one step, and apply distributive property, you get the matrix multiplication rule. http://math.stackexchange.com/questions/31725/intuition-behind-matrix-multiplication http://math.stackexchange.com/questions/31725/intuition-behi...
  - jperras 10y ago
    Axler's book, "Linear Algebra Done Right", is widely considered to be one of the best texts on linear algebra that focuses less on the mechanics and more on the structure of linear operators on vector spaces. http://linear.axler.net/ http://linear.axler.net/
    - CalChris 10y ago
      Yes, and the abridged version (no proofs, examples, and exercises) is downloadable for free. He avoids determinants altogether (well, until the 10th chapter). http://linear.axler.net/LinearAbridged.html http://linear.axler.net/LinearAbridged.html
77pt77 10y ago
In particular understanding that conjugation by another matrix is just a coordinate/frame transformation is crucial. Relying to much on explicit coordinates many times obfuscates what's going on.
- coherentpony 10y ago
  Is that true for non-square matrices?
  - lkasjdklasdj 10y ago
    Conjugation only makes sense for square matrices. More generally, left and right multiplication correspond to coordinate transformations of the output and input space, respectively.
    - coherentpony 10y ago
      If A is n x n and P is n x p then P is not square but P^T A P is well-defined.
samuell 10y ago
For understanding the mechanics of matrix multiplication, I found it useful to think in an analogy consisting of a grid of two layers of pipes; One -- the input-pipes -- coming in one direction, and the other -- the output pipes -- laid in an orthogonal direction. Then there would be "taps" in the cross-sections between the input and output pipes, representing the numbers (multiplication factors, really) in the matrix. I illustrated this in this little drawing: http://imgur.com/gallery/gBs64 http://imgur.com/gallery/gBs64 The point then is that the taps (again, representing the matrix values) determine how much of each item in the input vector, that should be mixed into each item in the output vector. This analogy has the limitation that the taps are allowed to enhance the flow, not just limit it, like physical taps would. That is, outputting more than 100% of the input :P Also, while this way of illustrating it may make some sense for matrix * vector multiplication, matrix * matrix would probably become a prohibitively cluttered image.
- snovv_crash 10y ago
  I really like that. And matrix * matrix would just have to be 3D, with a whole cube of taps. Sure, not easy to illustrate, but the concept extends well. It also makes it more obvious what the complexity of the operation is.
  - justinpombrio 10y ago
    In a little more detail: Look at the animation in the article, after the second matrix has been rotated and put on top of the first one. Then flip its top up so the two matrices are orthogonal. Finally, rotate the whole thing 90 degrees to the left (rotating along the axis that goes from the top of the page to the bottom of the page). Now you can see that the result will be a 2x3 matrix. And the values in each spot will be the sum of the products beneath it.
  - im3w1l 10y ago
    Actually the complexity of matrix multiplication is not known. Current best is n^2.37
    - 10y ago
ahane 10y ago
3Blue1Brown is an amazing youtube channel that provides very entertaining and educating visual explanation for all sorts of maths. He has a whole series on linear algebra, including matrix mulitplication: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...
- vanderZwan 10y ago
  ... and after a succesfull Patreon campaign[0], he's going to do "Essence of Calculus" next. I'm seriously excited! [0] https://www.patreon.com/3blue1brown https://www.patreon.com/3blue1brown
- daeken 10y ago
  I discovered this last night! I've been dealing with matrices frequently for the past 15 years and never really got them. Having watched his series now, I feel like I completely understand them, in a way that I never even fathomed before. I think I'm going to make a small series producing a raytracer from first principles, in a somewhat similar style; that visual design was just the perfect medium.
- bowlofstew 10y ago
  Thanks, I had not seen that channel before.
deleted 10y ago
[deleted]
nilkn 10y ago
This is why I'm a big fan of Axler's Linear Algebra Done Right. The book's emphasis is on the concepts behind the calculations rather than the calculations. I actually disagree with Axler on his avoidance of the determinant, though. I wish instead of avoiding it he'd spent more time developing it conceptually, as it's actually a fascinating construction. But to this day I have yet to find a gentler and better introduction to serious linear algebra than his book. He recommends it as a second course, but I read it during my first course in the subject and considered it my "secret weapon". I truly believe that book is what allowed me to get a perfect score in the class -- I had a conceptual understanding that was just not possible to glean from the official course textbook.
- pash 10y ago
  Axler's book is very good. An alternative to consider is Paul Halmos's much older Finite-Dimensional Vector Spaces [0]. Both books take the same basic approach, and the proofs of the major theorems are substantially the same. (Halmos's book is well known and well liked and was probably Axler's starting point.) Axler's book covers more ground (most notably, Halmos presents the polar decomposition but not the singular-value decompostion) and uses more modern terminology and notation. But Halmos's book has the merits of being half as long and a third as expensive, as well as having been written specifically to prepare the reader as directly as possible for Halmos's short introduction to Hilbert spaces [1]. I highly recommend one or the other of these books for readers who want to understand linear algebra as mathematicians do. 0. https://www.amazon.com/Finite-Dimensional-Vector-Spaces-Paul-Halmos/dp/178139573X/ https://www.amazon.com/Finite-Dimensional-Vector-Spaces-Paul... 1. https://www.amazon.com/Introduction-Hilbert-Theory-Spectral-Multiplicity/dp/1781395810/ https://www.amazon.com/Introduction-Hilbert-Theory-Spectral-...
  - ducttapecrown 10y ago
    Axler's book was free for me from my university's SpringerLink thingy.
    - kaeluka 10y ago
zubspace 10y ago
Is there something like this regarding homogeneous coordinates and the projective plane? It's strange. Somehow I have now problems whatsoever regarding the use of 4-component vectors. As long as the z component stays one, I'm all set. But lately I'm trying to understand the construction of projection matrices and how to apply the knowledge of homogeneous coordinates, for example shadow mapping. Parallel lines suddenly intersect? 3D space is just a projective plane? Now everything is all weird...
- ooqr 10y ago
  Homogenous coordinates have accrued really overcomplicated language around the very simple idea that something appears half as large twice as far away.
- thfuran 10y ago
  Not sure if you already are aware, but you didn't use the term, so I figured it might help to point out that what you are talking about is projective geometry. I'm not sure of any good texts, though it sounds like you might want one more focused on graphics than projective geometry purely in the abstract.
c3534l 10y ago
It at least cuts down on the amount of time you have to explain the mechanics of it. This is more intuitive to memorize than an algorithm composed of bullet-point instructions. For intuition, I highly recommend https://www.youtube.com/watch?v=kjBOesZCoqc https://www.youtube.com/watch?v=kjBOesZCoqc - I think both the intuition and practicing the details should be learned.
lisper 10y ago
You'd probably like this then: https://graphicallinearalgebra.net https://graphicallinearalgebra.net

bandrami 10y ago

Bingo. We did matrices in high school and even through a first linear algebra course in undergrad I got more or less competent at the mechanics of matrix multiplication without ever having any sense of why anyone would want to do this or why these rules, as opposed to some other set of rules that were equally formally valid, were of any interest. I could even tell you what the eigenvectors of a matrix were but had no sense of what that actually meant in terms of a linear transformation that matrix represented. Linear algebra is probably the area of math that is both universally important and completely skimped on by our education system.

t3nary 10y ago
I feel very similar. Do you have any recommendations for resources to learn about why matrix multiplication is defined that way?
- darrickw 10y ago
  Check out this series of videos "The Essence of Linear Algebra"[1] for a really powerful visual and intuitive explanation. It starts with vectors and builds to matrix multiplication and further to several other topics. [1]: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...
- bandrami 10y ago
  "All the mathematics you missed but need to know for graduate school"[1] helped me a lot (and, in fact, I had and did). Once I had finished that, Cullen's "Matrices and linear transformations"[2] was really helpful too. But I wouldn't do Cullen if you're still, as I was, floundering with the concepts of why you're doing this in the first place. It's great once you have those concepts down. [1]: https://www.amazon.com/All-Mathematics-You-Missed-Graduate/dp/0521797071 https://www.amazon.com/All-Mathematics-You-Missed-Graduate/d... [2]: http://store.doverpublications.com/0486663280.html http://store.doverpublications.com/0486663280.html
  - fsloth 10y ago
    Garrity's book looks exactly what I've been looking for for brushing up on basic math skills after a decade of fairly low-brow work after my MSc. Thanks for the reference!
    - bandrami 10y ago
      Hope it helps! It definitely got me through the first year of grad school...
- cousin_it 10y ago
  Let's say you have a system with N possible states, evolving in discrete steps. At each step, the system has some probability of switching from any state to any other state. That gives you an NxN matrix of switching probabilities. For example, if the system always stays in the same state as it started, the switching probabilities are an identity matrix (1 on the diagonal and 0 everywhere else). Now let's see what happens after two steps. If the system started out in state i, what's the probability that after two steps it will end up in state k? Well, it's the sum over all possible paths. In other words, the sum of probabilities of i->j->k for all possible j. In other words, the sum of p_{ij} times p_{jk} for j from 1 to N. But that's exactly the definition of multiplying a matrix by itself. Now it should be easy to understand that whenever you have matrices that represent transformations of some object, composing transformations will correspond to multiplying matrices.
- ww2 10y ago
  If you compose two linear transformations into one step, and apply distributive property, you get the matrix multiplication rule. http://math.stackexchange.com/questions/31725/intuition-behind-matrix-multiplication http://math.stackexchange.com/questions/31725/intuition-behi...
- jperras 10y ago
  Axler's book, "Linear Algebra Done Right", is widely considered to be one of the best texts on linear algebra that focuses less on the mechanics and more on the structure of linear operators on vector spaces. http://linear.axler.net/ http://linear.axler.net/
  - CalChris 10y ago
    Yes, and the abridged version (no proofs, examples, and exercises) is downloadable for free. He avoids determinants altogether (well, until the 10th chapter). http://linear.axler.net/LinearAbridged.html http://linear.axler.net/LinearAbridged.html

77pt77 10y ago

In particular understanding that conjugation by another matrix is just a coordinate/frame transformation is crucial. Relying to much on explicit coordinates many times obfuscates what's going on.

coherentpony 10y ago
Is that true for non-square matrices?
- lkasjdklasdj 10y ago
  Conjugation only makes sense for square matrices. More generally, left and right multiplication correspond to coordinate transformations of the output and input space, respectively.
  - coherentpony 10y ago
    If A is n x n and P is n x p then P is not square but P^T A P is well-defined.

samuell 10y ago

For understanding the mechanics of matrix multiplication, I found it useful to think in an analogy consisting of a grid of two layers of pipes; One -- the input-pipes -- coming in one direction, and the other -- the output pipes -- laid in an orthogonal direction. Then there would be "taps" in the cross-sections between the input and output pipes, representing the numbers (multiplication factors, really) in the matrix. I illustrated this in this little drawing: http://imgur.com/gallery/gBs64 http://imgur.com/gallery/gBs64 The point then is that the taps (again, representing the matrix values) determine how much of each item in the input vector, that should be mixed into each item in the output vector. This analogy has the limitation that the taps are allowed to enhance the flow, not just limit it, like physical taps would. That is, outputting more than 100% of the input :P Also, while this way of illustrating it may make some sense for matrix * vector multiplication, matrix * matrix would probably become a prohibitively cluttered image.

snovv_crash 10y ago
I really like that. And matrix * matrix would just have to be 3D, with a whole cube of taps. Sure, not easy to illustrate, but the concept extends well. It also makes it more obvious what the complexity of the operation is.
- justinpombrio 10y ago
  In a little more detail: Look at the animation in the article, after the second matrix has been rotated and put on top of the first one. Then flip its top up so the two matrices are orthogonal. Finally, rotate the whole thing 90 degrees to the left (rotating along the axis that goes from the top of the page to the bottom of the page). Now you can see that the result will be a 2x3 matrix. And the values in each spot will be the sum of the products beneath it.
- im3w1l 10y ago
  Actually the complexity of matrix multiplication is not known. Current best is n^2.37
  - 10y ago

ahane 10y ago

3Blue1Brown is an amazing youtube channel that provides very entertaining and educating visual explanation for all sorts of maths. He has a whole series on linear algebra, including matrix mulitplication: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...

vanderZwan 10y ago
... and after a succesfull Patreon campaign[0], he's going to do "Essence of Calculus" next. I'm seriously excited! [0] https://www.patreon.com/3blue1brown https://www.patreon.com/3blue1brown
daeken 10y ago
I discovered this last night! I've been dealing with matrices frequently for the past 15 years and never really got them. Having watched his series now, I feel like I completely understand them, in a way that I never even fathomed before. I think I'm going to make a small series producing a raytracer from first principles, in a somewhat similar style; that visual design was just the perfect medium.
bowlofstew 10y ago
Thanks, I had not seen that channel before.

deleted 10y ago

[deleted]

nilkn 10y ago

This is why I'm a big fan of Axler's Linear Algebra Done Right. The book's emphasis is on the concepts behind the calculations rather than the calculations. I actually disagree with Axler on his avoidance of the determinant, though. I wish instead of avoiding it he'd spent more time developing it conceptually, as it's actually a fascinating construction. But to this day I have yet to find a gentler and better introduction to serious linear algebra than his book. He recommends it as a second course, but I read it during my first course in the subject and considered it my "secret weapon". I truly believe that book is what allowed me to get a perfect score in the class -- I had a conceptual understanding that was just not possible to glean from the official course textbook.

pash 10y ago
Axler's book is very good. An alternative to consider is Paul Halmos's much older Finite-Dimensional Vector Spaces [0]. Both books take the same basic approach, and the proofs of the major theorems are substantially the same. (Halmos's book is well known and well liked and was probably Axler's starting point.) Axler's book covers more ground (most notably, Halmos presents the polar decomposition but not the singular-value decompostion) and uses more modern terminology and notation. But Halmos's book has the merits of being half as long and a third as expensive, as well as having been written specifically to prepare the reader as directly as possible for Halmos's short introduction to Hilbert spaces [1]. I highly recommend one or the other of these books for readers who want to understand linear algebra as mathematicians do. 0. https://www.amazon.com/Finite-Dimensional-Vector-Spaces-Paul-Halmos/dp/178139573X/ https://www.amazon.com/Finite-Dimensional-Vector-Spaces-Paul... 1. https://www.amazon.com/Introduction-Hilbert-Theory-Spectral-Multiplicity/dp/1781395810/ https://www.amazon.com/Introduction-Hilbert-Theory-Spectral-...
- ducttapecrown 10y ago
  Axler's book was free for me from my university's SpringerLink thingy.
  - kaeluka 10y ago

zubspace 10y ago

Is there something like this regarding homogeneous coordinates and the projective plane? It's strange. Somehow I have now problems whatsoever regarding the use of 4-component vectors. As long as the z component stays one, I'm all set. But lately I'm trying to understand the construction of projection matrices and how to apply the knowledge of homogeneous coordinates, for example shadow mapping. Parallel lines suddenly intersect? 3D space is just a projective plane? Now everything is all weird...

ooqr 10y ago
Homogenous coordinates have accrued really overcomplicated language around the very simple idea that something appears half as large twice as far away.
thfuran 10y ago
Not sure if you already are aware, but you didn't use the term, so I figured it might help to point out that what you are talking about is projective geometry. I'm not sure of any good texts, though it sounds like you might want one more focused on graphics than projective geometry purely in the abstract.

c3534l 10y ago

It at least cuts down on the amount of time you have to explain the mechanics of it. This is more intuitive to memorize than an algorithm composed of bullet-point instructions. For intuition, I highly recommend https://www.youtube.com/watch?v=kjBOesZCoqc https://www.youtube.com/watch?v=kjBOesZCoqc - I think both the intuition and practicing the details should be learned.

lisper 10y ago

You'd probably like this then: https://graphicallinearalgebra.net https://graphicallinearalgebra.net