9 ms·

Matrix Multiplication

873 points by kasbah 10y ago

rockdoe 10y ago
Misrenders on Firefox for Android.
- quickben 10y ago
  Looks fine on mine, what's the Android/Firefox version there?
- staltz 10y ago
  Yeah, I'm aware. It annoys me too. https://github.com/staltz/matrixmultiplication.xyz/issues/1 https://github.com/staltz/matrixmultiplication.xyz/issues/1
hodder 10y ago
Cool visualization.
shultays 10y ago
Change to color of rows to red & blue (while multiplying) so it is easier to which result is which product.
- kejaed 10y ago
  This is a great idea
gregn610 10y ago
Nice. I needed something like that when I was getting started with ML courses.
cousin_it 10y ago
I think a cool way to visualize matrix multiplication would be to put three matrices MxN, NxK, MxK on the sides of a three-dimensional box MxNxK. Each cell of the matrix MxK would cast a "shadow" onto some row in the MxN matrix and some column in the NxK matrix, whose dot product would give the value of that cell.
- moultano 10y ago
  That's exactly how I finally fixed it in my head permanently.
- kakarot 10y ago
  Would you mind elaborating or providing a link for the uninitiated? I use matrix multiplication in almost everything I do and that sounds like a neat way to visualize what is happening.
daw___ 10y ago
Awesome. What about making it more clear that the "End" button will animate remaining steps? Before clicking it I thought it would have jumped to the last step.
IshKebab 10y ago
This is nearly how I think of it, but for me I do one column of the second matrix at a time. I think it is clearer that way. This way makes it look like there is some kind of staircase thing going on which there isn't really.
- nickcw 10y ago
  That was my thought too - I was taught to do one row of the first matrix with one column of the second matrix at a time. I found the whole staircase thing quite confusing.
zubspace 10y ago
I like the idea, but my brain is wired differently. Usually, when I do matrix multiplications, the rows on the left side transition to the columns on the right side. In the example, the 1 goes to 2, 2 to 6, 1 to 1. The animation completely confused me, because it looks like being reversed, but the result is the same. Maybe, make this a visualization option?
- denzil_correa 10y ago
  Essentially, that is how I was taught and do matrix multiplication as well. But this sliding window approach is also interesting. Although, it would be a bit complex to understand how to fill in the results based on the sliding window.
- kutkloon7 10y ago
  I do this too and this is way more intuitive in my opinion, as you can easily see which entries of the resulting matrix are linear in which rows and columns. Also, you don't need to transform the matrices in a way that is impossible to do on paper.
- conjectures 10y ago
  Me too. This is like seeing someone write legible English starting from the right hand side of the paper. Not wrong, but discordant with my expectations.
- nkurz 10y ago
  There are several other visualizations that can be useful too: Multiplication We can see matrix by matrix multiplication from 5 different positions: row by column multiplication column at a time row at a time as sum of outer products block multiplication All of them are equivalent and lead to the same result http://mlwiki.org/index.php/Matrix-Matrix_Multiplication http://mlwiki.org/index.php/Matrix-Matrix_Multiplication Does your mental model fit into one of these, or is it also reversed in the order it is applied? It would indeed be interesting to allow visualizations of all the approaches.
Jaruzel 10y ago
I have never understood Matrix Multiplication - which has always prevented me from doing cool 3D stuff. Now I do! THANK YOU! :)
- popcorncolonel 10y ago
  The power of visualization.
- clarry 10y ago
  You totally don't need matrices to do cool 3D stuff though.
known 10y ago
Cool
NumberCruncher 10y ago
I wish we had visualizations like this back in the 90's when I studied statistics.
kasbah 10y ago
I wonder if this visualization could be turned into a fun game.
daniel-levin 10y ago
This is a cool example of what Bret Victor calls an "Explorable Explanation" [0]. That said, I feel that it's more important to understand how and why matrix multiplication corresponds to a composition of linear transformations than learning the actual mechanics of doing the computation. You can get good at matrix multiplication without knowing what is going on. I view that as a less valuable activity than learning about linear transformations (vector-space structure preserving mappings) between finite dimensional vector spaces. [0] http://worrydream.com/ExplorableExplanations/ http://worrydream.com/ExplorableExplanations/
- bandrami 10y ago
  Bingo. We did matrices in high school and even through a first linear algebra course in undergrad I got more or less competent at the mechanics of matrix multiplication without ever having any sense of why anyone would want to do this or why these rules, as opposed to some other set of rules that were equally formally valid, were of any interest. I could even tell you what the eigenvectors of a matrix were but had no sense of what that actually meant in terms of a linear transformation that matrix represented. Linear algebra is probably the area of math that is both universally important and completely skimped on by our education system.
  - t3nary 10y ago
    I feel very similar. Do you have any recommendations for resources to learn about why matrix multiplication is defined that way?
    - darrickw 10y ago
      Check out this series of videos "The Essence of Linear Algebra"[1] for a really powerful visual and intuitive explanation. It starts with vectors and builds to matrix multiplication and further to several other topics. [1]: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...
    - bandrami 10y ago
lima 10y ago
Appropriate use of the .xyz TLD :-)
joshaidan 10y ago
Interesting, I wonder why Facebook has this website blocked, preventing me from posting it on my wall.
- staltz 10y ago
  Good sign that it's time to leave Facebook. Really
- roywiggins 10y ago
  Maybe the slightly unusual TLD?
- geonnave 10y ago
  I just had the same problem! I wanted to share it with the College group from where I graduated, but I could not do it.
mfirmin 10y ago
This reminds me of Nicky Case's Explorable Explanation of a 2D transformation matrix: http://ncase.me/matrix/ http://ncase.me/matrix/
wodenokoto 10y ago
As others have mentioned, many introductions to linear algebra / matrix multiplication only gives the student a formula, but not why the formula looks like it does. Why is matrix multiplication not pointwise, was something I had a hard time understanding. Better explained has a good article on this, that really made it click for me https://betterexplained.com/articles/linear-algebra-guide/ https://betterexplained.com/articles/linear-algebra-guide/
hacker_9 10y ago
As a gamedev who uses matrices all the time, I don't find this animation useful at all for practical work. Instead I think of matrix multiplication as creating a list of ordered operations (scale x translate x rotate etc) which is just encoded efficiently in a table, to then be sent to other parts of the program for use. You can even remove items from the list by multiplying by their inverse.
- scraft 10y ago
  But of course when you have encoded the scale, rotate and translate into the matrix, it isn't possible to decode it back out to your original values. This is obvious if you consider the fact the following produce identical matrices (just consider 2D for now): 1. Scale in X and Y of -1 2. Z rotation of 180
  - hacker_9 10y ago
    Yes, I know, what is your point exactly? When I say 'list of ordered operations' I mean that if I were to apply the matrix to any object, you can imagine each transform in the list being applied one after another to the object visually, resulting in the final position.
FeepingCreature 10y ago
This is sounding increasingly like the Monad situation, where everybody has their own incompatible approach for explaining it, and none of them make any sense. I feel like I understand Matrices _less_ now than when I started reading around here.
- maxiepoo 10y ago
  If by "Monad situation" you mean trying to learn a concept by just reading a single blog post then sure. Trying to learn something by scattered blog posts without engaging in material is a bad idea no matter what the problem is. For some reason nobody expects to read a blog post and understand the fundamental theorem of calculus without first learning intuition and doing calculations with derivatives and integrals. On the other hand, if you have been working through a concept and you read a blog post that gives a different perspective than you're used to, it might help. If you want to actually learn something, then get something like a book or series of posts and work through examples and explore the ideas yourself. But having more than one intuition is useful and complaining about the fact that there is more than one way to understand something is lazy.
thanatropism 10y ago
See the very first video in Strang's MIT Linear Algebra course on YouTube. There's two ways of seeing matrix multiplication (from a matrix POV, rather than as linear transformations).
adamnemecek 10y ago
I have found the Stanford EE263 class to be possibly the best class on linear algebra. This might be due to the fact that it's technically a class about linear dynamic systems not linear algebra per se http://ee263.stanford.edu/ http://ee263.stanford.edu/
user1713952 10y ago
The idea is good but I am afraid it misses the point entirely. Multiplying both columns of the right hand side matrix to the left hand side matrix is really not helping the target audience understand what is going on. 2 operations are going on as the same time. I believe https://www.youtube.com/watch?v=XkY2DOUCWMU https://www.youtube.com/watch?v=XkY2DOUCWMU is a much better way to illustrate the matrix multiplication (and provide a clear explanation of the why) .
- ahane 10y ago
  Beat me to it :) amazing channel!
- staltz 10y ago
  Comments like these (aside from providing a nice link) just make authors of content sad. Why say such thing? You can say "here is another helpful link", but "misses the point entirely"? The point is to provide a self-evident mnemonic, as explained here: https://github.com/staltz/matrixmultiplication.xyz https://github.com/staltz/matrixmultiplication.xyz Not nice.
  - user1713952 10y ago
    Mr Staltz, please allow me to elaborate. I did not mean to be mean at all, nor to reduce the value of your work, excuse me if I made you feel like that. I am actually grateful that you made this animation as it allows one to reflect on the concept. If your kid proudly brings home a drawing, you are most probably going to greet him with warm encouragements no matter the quality of the art. The kid is in a process of developing his creative sense, learning, and forming his/her sense of reality. Comparing the drawing to what mature artists produce would certainly have a negative effect on the kid. The point is to provide a feedback relevant to the recipient of the critic. The same goes here with this animation. I respect your work but obviously disagree with the usefulness of it. You made an animation to explain the matrix product, which is a really good idea. Alas, I think the realization of it gives a confused picture of what it is. I doubt that a student with no prior understanding of matrix products will get much enlightment. Too many operations are going on at the same time. This means that it misses its intended audience: students in need of building an intuitive understanding of the matrix product. That's what all those javascript (among others) animations (ex: http://jackschaedler.github.io/circles-sines-signals/dft_introduction.html http://jackschaedler.github.io/circles-sines-signals/dft_int...) are all about: make it simpler to grasp a concept. Unfortunately, though beautiful, the animation is not clear. At best it provides a visual mnemonic for multiplying matrices which maybe useful for some but not for the younger version of me in need of understanding :) I realize that instead of providing such a harsh critic, I would have been better inspired to suggest improvements. Mea culpa :)
coin 10y ago
Uggg, why is zoom disable on mobile devices?
Vinkekatten 10y ago
Meh, the site is blocked at my workplace. Websense is really crummy.
esalman 10y ago
Another one for matrix inversion would be nice.
ww2 10y ago
I think this is overdone and does not help the intuition. two things: (1). the first step of transposing the second matrix 90 degree is unnecessary and confusing. A row vector and a column vector are different. This step confuses the two. (2). All the final elements are independent and should be calculated in parallel, the stepped anime is not necessary and does not reveal this basic fact.
inglor 10y ago
Question for staltz - are you happy with the cycle code at https://github.com/staltz/matrixmultiplication.xyz/ https://github.com/staltz/matrixmultiplication.xyz/ ? It feels like a lot of work for this sort of task which I feel could have been achieved by using an existing (mutable) matrix lib and drawing ad-hoc. It feels like an awful lot of code written - especially with all the `.something === "SOME_STRING_CONSTANT"` (couldn't those be typed directly in TS btw?). Still, you're a smart guy - I'm trying to see why it's worth it. Do you really feel model-view-intent shines here? (Nice visualization by the way)
- staltz 10y ago
  Yes, I'm very happy with that code, apart from one thing: how animations are done sometimes with tweens in JS, and sometimes with CSS. I'm sure it could have been built with less code. In general, the more shortcuts you take and the less abstraction you use, the less amount of code you will have. But also, it's likely that you would mix concerns more easily. So in general, when separating concerns, you end up getting more LOCs. That's what I've perceived in my experience, not a fact that I can prove. So I do feel like model-view-intent shines here. Model only has state concerns. I was careful in moving away DOM-related code from the model. So theoretically you could reuse this model for React Native or any other UI target. View only displays state in a visual representation, and intent only captures user events and represents them in a form suitable for the model. About some TypeScript code, I think you referred to https://github.com/staltz/matrixmultiplication.xyz/blob/master/src/Calculator/model/actions.ts#L58-L80 https://github.com/staltz/matrixmultiplication.xyz/blob/mast.... These could have been avoided. The only purpose they serve is to make inference work out for lines like these: https://github.com/staltz/matrixmultiplication.xyz/blob/master/src/Calculator/model/reducers/controlPanel.ts#L16-L17 https://github.com/staltz/matrixmultiplication.xyz/blob/mast... TypeScript knows that `action` is of type StartMultiplyAction. If we had only done `.filter(ac => ac.type === 'START_MULTIPLY')`, TypeScript couldn't infer anymore. So I understand those isFoo functions are very boilerplatey, but they serve the purpose to provide accurate typings. It's optional and depending how you write TypeScript code, you can use more type casting and more `any`, or you can go down the route of providing accurate types everywhere, which is what I did. This is not at all related to Cycle.js nor MVI. Feel free to ask more architecture-related questions. I like these. What else feels like "too much code"?
hdivider 10y ago
Awesome. Some concepts in math are so much easier to grasp with an interactive visual. I once made an entire game around matrix multiplication -- such that players didn't even know they learned how to do this. And sometimes, visualizations can help cause discoveries. A lot of progress in theoretical physics came about because physicists 'guessed' what nature's equations should look like...and then got shocked when they discovered they were right. Take James Clerk Maxwell, for instance: "He had achieved his greatest success by relying on just one empirical fact known to every school boy. Yet now he tried to do without any empirical facts, by pure thinking. He believed in the power of reason to guess the laws according to which God has built the world." http://www.mathpages.com/home/kmath103/kmath103.htm http://www.mathpages.com/home/kmath103/kmath103.htm
- abecedarius 10y ago
  I'm curious about this game. Is it online?
  - hdivider 10y ago
    I built it for the Windows 8 Store (of all places), years ago, in base C++/Cx. All de-published now though. Could dig up a gameplay video if you wish...
    - abecedarius 10y ago
      Thanks, but I'm just curious and don't like videos. The world needs more things like that, though -- it was nice to hear of it.
amelius 10y ago
How I would explain it: - First, show how to multiply a row vector by a column vector. - Then, show how to "simultaneously" multiply several row vectors by a single column vector by placing the row vectors in a matrix. - Now, similarly, do the same with the column vector. And voila, matrix by matrix multiplication.
- appleflaxen 10y ago
  im trying to understand this, and what you said appeals to me but i can't quite visualize what you mean. do you have an illustration?
jakehoon 10y ago
This would have been so awesome to have in college. I stumbled upon RegExr the other day (http://regexr.com http://regexr.com) which I absolutely love. It kind of shares this spirit of showing people how to do things that appear complex at first glance, but have a sort of addicting nuance to them. Thanks for sharing.
rsp1984 10y ago
Very nice, but the mental model I prefer is to think of a Matrix-Vector multiplication as the weighted sum of Matrix columns, and a Matrix-Matrix multiplication as just several Matrix-Vector multiplications (results being the columns of a new matrix).
tskaiser 10y ago
This is cool, but I have a few criticisms: - Flipping something over like that looks cool, but confuses the brain. At least it confused mine. - Inconsistently calculating 1 or 2 elements at a time is confusing. Having 2 results calculated at a time simply put requires the recipient to untangle what is going on. This only gets worse when the recipient tries to do it themselves for examples with more columns in the right matrix. Disclaimer: I am biased as I already know how to do matrix operations, and I visualize it differently. When multiplying by hand I translate the right matrix up, like in the animation, but do not flip it over. Then I calculate each element one at a time by taking the sum of products of the row and column that intersects over that element. Like so: http://i.imgur.com/D572zOI.png http://i.imgur.com/D572zOI.png As you can see I'm not much of a graphics artist. Edit: actually, I think this would be really good if you "filter down" one column at a time, instead of moving the whole matrix down, so that it only calculates one element at a time, one column at a time. Means the animation will contain more steps, but I think that is actually to its benefit. Does this make sense?
- qb45 10y ago
  > Flipping something over like that looks cool, but confuses the brain. At least it confused mine. I think it would be helpful to have a shadow of the right matrix after it's moved to the top. This way it would be easier to see where particular numbers are coming from in the original input. Disclaimer: I'm similarly biased and actually the reason I came up with this shadow thing is because I also imagine the right matrix moving like here but somehow this never causes it to disappear from its original place on paper, go figure, so this animation looked really alien to me.
ryanmonroe 10y ago
For me personally, a simple animated version of this image[0] looping through rows of A and columns of B seems easier to understand. [0] https://upload.wikimedia.org/wikipedia/commons/e/eb/Matrix_multiplication_diagram_2.svg https://upload.wikimedia.org/wikipedia/commons/e/eb/Matrix_m...
j1vms 10y ago
Not sure if anyone here has mentioned this yet, but it's useful to think of matrix multiplication as the most natural extension of one-dimensional algebra into multiple dimensions. First, remember that people first wrestled with and found solutions to "1-D" equations like "ax^2 + bx + c = 0". When others came along asking about how to extend these results into multiple dimensions so that previous results might still be useful, the most natural answer was found to be how we now define matrix multiplication. To see further evidence of this, take your favorite linear algebraic equation using matrices and then look at the edge case where you have only 1x1 matrices in your equation. Usually you'll end up with something that looks analogous to a result in basic algebra in real number (or complex) variables, not matrices. If all this still seems counter-intuitive, then realize this is essentially why math is so important to our logic. It helps us to reason about things that are not obvious, yet logically must fall through from the basic tautologies we "chose" (like 0 != 1, addition on integer numbers, etc.). Or, more aptly in this case, it helps retain an analogous symbolic representation (the overall logic of the equation at the high level) despite a change in the underlying nuts and bolts of computing solutions to it (matrix multiplication vs. our regular "1D" multiplication).
- gunnihinn 10y ago
  I'm sorry, but that's not how that works. Your example is of a second-degree equation, which can't be solved by linear algebra. Linear algebra was invented to solve systems of linear equations, and matrices first arose as a convenient notation for expressing those. Matrix multiplication came later and is now understood to correspond to the composition of linear morphisms on vector spaces. If you try to extend the (very few) methods we have for obtaining exact solutions to polynomial equations in one variable to higher dimension, you (a) fail and (b) invent algebraic geometry.
jayajay 10y ago
It's easier to think of matrix multiplication by computing the matrix elements, which means reducing the problem to NM vector dot products. To get the element of the result matrix at position (n,m), compute the inner product < r_n | c_m >, where r_n is the nth row vector of the left matrix and c_m is the mth column vector of the right matrix. Once you try it, you'll see that it's also much easier to visualize than this strange, unintuitive approach. With great power comes great responsibility. That domain name belongs to that person. I hope they do society a favor and put more content regarding MM that that, because they chose one of the least intuitive ways of thinking about it. Put more ways up, or ditch the domain. If the goal is to get people to understand what matrices are, the best way is to teach them about operators, transformations, vector spaces and linear algebra in general, because this is really the only way to fully understand what's going on without relying on some heuristic. If the goal is to get people to remember how to do matrix multiplication, at least put up more ways of doing it.
- staltz 10y ago
  Nope, won't ditch the domain. I'd recommend you ditch that type of speech.
  - deleted 10y ago
    [deleted]
  - 0xFFFFFF 10y ago
    Everyone's a critic, don't let it get to you.
  - jayajay 10y ago
    I'm sorry if my words upset you. That said, it would be a shame to tell someone off if they bring up a good point (however brash). I used to teach this stuff, and I can tell you many of my students would end up being more confused with your visualization. I hope you can see that you've assumed some responsibility. That's all I can hope.
cocktailpeanuts 10y ago
I have no idea what's going on there. But it's a cool piece of art, I'm guessing that was the intention.
jordigh 10y ago
I find this image a lot easier to work with. The animation above just seems a tad confusing: https://staff.fnwi.uva.nl/r.vandenboomgaard/IPCV/_images/Matrix_multiplication_diagram_2.png https://staff.fnwi.uva.nl/r.vandenboomgaard/IPCV/_images/Mat...
tvural 10y ago
Matrices are linear maps, and matrix multiplication composes the linear maps. Now statements like the determinant of the product is the product of the determinants, the trace is preserved under similarity transforms (since similarity transforms just rewrite the linear map in a different basis), etc. are intuitive. I've always felt that these explicit calculations don't really get to the point. You can memorize them and still not really understand what's going on.
- HQvIdsGkVw 10y ago
  When I was a math student, we had two semesters of linear algebra. We only saw matrices at the end of the first semester; other than that it was all about linear mappings. The thing is, most linear algebra classes that are taught to other sections (biology, CS, …) start directly with matrices and are mostly computational.
- gunnihinn 10y ago
  Maybe you know something I don't, but I don't find these statements about the determinant and trace intuitive just from knowing that matrix multiplication corresponds to composition of linear maps. They can be made intuitive by appealing to geometry, but I don't see how to do that just by staring at the algebraic picture. Actually, proving them without picking a basis and then showing that what you get is invariant under base change is quite nontrivial and involves throwing a lot of heavy machinery around; see for example Coffman's non-coordinate proof of the statement about the trace in http://users.ipfw.edu/CoffmanA/pdf/book.pdf http://users.ipfw.edu/CoffmanA/pdf/book.pdf For the determinant, if you want to do everything without picking a basis, you're basically proving that the exterior power operation defines a functor on the category of finite-dimensional vector spaces, which isn't that bad to do, but you somehow have to explain what all those words mean along the way.
scalablenotions 10y ago
Thank you! This is marvelous and really made me understand this process at a much simpler level. Please consider making more for other mathematical concepts :)
HeavyStorm 10y ago
I read the title and thought, oh God, not another one, the second was bad enough and I like to pretend that the third never happened...
matteuan 10y ago
Nice, now let's do it with the Strassen's algorithm
JabavuAdams 10y ago
Neat visualization. I feel it's important and eye-opening to note that there's not just one way to multiply matrices, there're five (according to Strang): 5 ways to multiply A B = C 1) Each element of C is a dot-product of corresponding rows of A with columns of B 2) Each column of C is a combination of the columns A. Each column of B has the coefficients. 3) Each row of C is a combination of the rows of B. Each row of A has the coefficients. 4) Accumulate outer-product. Accumulate sums of of outer-products of each col of A with each row of B. 5) Block multiplication They all produce the same answer, but different ones provide different insight for particular situations. See Video Lecture #3 of Strang's Linear Algebra course for more info: https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/lecture-3-multiplication-and-inverse-matrices/ https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra...
- jasonx1e 10y ago
  His lectures are awesome, wish some of them could be remastered though
mrcactu5 10y ago
this is more than just visualization. i can touch the computation. each step is tangible
jasonx1e 10y ago
Now make one that can find inverses ( ͡° ͜ʖ ͡°)
kafkaesq 10y ago
I my head I see the matrices remaining in place, while the vectors from the left slowly float to the right, produce dot products, which in turn float into their destination cells in the new output matrix just to the right of the equals sign. But that's just me.
greggman 10y ago
This is very cool. For me I needed to know why it works not what it does. I worked it out here for 2d transformations http://webglfundamentals.org/webgl/lessons/webgl-2d-matrices.html http://webglfundamentals.org/webgl/lessons/webgl-2d-matrices... and then expanded it to 3d, perspective, and cameras.
ivan_ah 10y ago
Nice illustration of the "row picture" of matrix multiplication. The "column picture" is equally important. This intro lecture by Prof. Strang is a really good lesson on that: https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/lecture-1-the-geometry-of-linear-equations/ https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra... Also, this short tutorial on linear algebra covers the row + column pictures on the first page: https://minireference.com/static/tutorials/linear_algebra_in_4_pages.pdf https://minireference.com/static/tutorials/linear_algebra_in... (disclaimer: I'm the author)
vladsotirov 10y ago
The UI confuses me to the point of frustration. 1. A "Previous" button would help since otherwise you have to go loop through the whole thing to see what actually happened (e.g. how exactly the first step lifts and rotates the matrix, what's getting added to what). 2. The final animation step is a transition from the patently false equation "[matrix1]=[result_matrix]" to the finally correct equation "[matrix1] x [matrix2] = [result_matrix]". This could be avoided if instead of transforming "[matrix1] x [matrix2]" into "[matrix1 plus overlay]=[result_matrix to be filled]" and then doing the 'waterfall', the equation "[matrix1] x [matrix2] = [result_matrix to be filled]" was maintained throughout, perhaps with the moving overlay slightly grayer after lifting [matrix2].

873 points by kasbah 10y ago

rockdoe 10y ago

Misrenders on Firefox for Android.

quickben 10y ago
Looks fine on mine, what's the Android/Firefox version there?
staltz 10y ago
Yeah, I'm aware. It annoys me too. https://github.com/staltz/matrixmultiplication.xyz/issues/1 https://github.com/staltz/matrixmultiplication.xyz/issues/1

hodder 10y ago

Cool visualization.

shultays 10y ago

Change to color of rows to red & blue (while multiplying) so it is easier to which result is which product.

kejaed 10y ago
This is a great idea

gregn610 10y ago

Nice. I needed something like that when I was getting started with ML courses.

cousin_it 10y ago

I think a cool way to visualize matrix multiplication would be to put three matrices MxN, NxK, MxK on the sides of a three-dimensional box MxNxK. Each cell of the matrix MxK would cast a "shadow" onto some row in the MxN matrix and some column in the NxK matrix, whose dot product would give the value of that cell.

moultano 10y ago
That's exactly how I finally fixed it in my head permanently.
kakarot 10y ago
Would you mind elaborating or providing a link for the uninitiated? I use matrix multiplication in almost everything I do and that sounds like a neat way to visualize what is happening.

daw___ 10y ago

Awesome. What about making it more clear that the "End" button will animate remaining steps? Before clicking it I thought it would have jumped to the last step.

IshKebab 10y ago

This is nearly how I think of it, but for me I do one column of the second matrix at a time. I think it is clearer that way. This way makes it look like there is some kind of staircase thing going on which there isn't really.

nickcw 10y ago
That was my thought too - I was taught to do one row of the first matrix with one column of the second matrix at a time. I found the whole staircase thing quite confusing.

zubspace 10y ago

I like the idea, but my brain is wired differently. Usually, when I do matrix multiplications, the rows on the left side transition to the columns on the right side. In the example, the 1 goes to 2, 2 to 6, 1 to 1. The animation completely confused me, because it looks like being reversed, but the result is the same. Maybe, make this a visualization option?

denzil_correa 10y ago
Essentially, that is how I was taught and do matrix multiplication as well. But this sliding window approach is also interesting. Although, it would be a bit complex to understand how to fill in the results based on the sliding window.
kutkloon7 10y ago
I do this too and this is way more intuitive in my opinion, as you can easily see which entries of the resulting matrix are linear in which rows and columns. Also, you don't need to transform the matrices in a way that is impossible to do on paper.
conjectures 10y ago
Me too. This is like seeing someone write legible English starting from the right hand side of the paper. Not wrong, but discordant with my expectations.
nkurz 10y ago
There are several other visualizations that can be useful too: Multiplication We can see matrix by matrix multiplication from 5 different positions: row by column multiplication column at a time row at a time as sum of outer products block multiplication All of them are equivalent and lead to the same result http://mlwiki.org/index.php/Matrix-Matrix_Multiplication http://mlwiki.org/index.php/Matrix-Matrix_Multiplication Does your mental model fit into one of these, or is it also reversed in the order it is applied? It would indeed be interesting to allow visualizations of all the approaches.

Jaruzel 10y ago

I have never understood Matrix Multiplication - which has always prevented me from doing cool 3D stuff. Now I do! THANK YOU! :)

popcorncolonel 10y ago
The power of visualization.
clarry 10y ago
You totally don't need matrices to do cool 3D stuff though.

known 10y ago

Cool

NumberCruncher 10y ago

I wish we had visualizations like this back in the 90's when I studied statistics.

kasbah 10y ago

I wonder if this visualization could be turned into a fun game.

daniel-levin 10y ago

This is a cool example of what Bret Victor calls an "Explorable Explanation" [0]. That said, I feel that it's more important to understand how and why matrix multiplication corresponds to a composition of linear transformations than learning the actual mechanics of doing the computation. You can get good at matrix multiplication without knowing what is going on. I view that as a less valuable activity than learning about linear transformations (vector-space structure preserving mappings) between finite dimensional vector spaces. [0] http://worrydream.com/ExplorableExplanations/ http://worrydream.com/ExplorableExplanations/

bandrami 10y ago
Bingo. We did matrices in high school and even through a first linear algebra course in undergrad I got more or less competent at the mechanics of matrix multiplication without ever having any sense of why anyone would want to do this or why these rules, as opposed to some other set of rules that were equally formally valid, were of any interest. I could even tell you what the eigenvectors of a matrix were but had no sense of what that actually meant in terms of a linear transformation that matrix represented. Linear algebra is probably the area of math that is both universally important and completely skimped on by our education system.
- t3nary 10y ago
  I feel very similar. Do you have any recommendations for resources to learn about why matrix multiplication is defined that way?
  - darrickw 10y ago
    Check out this series of videos "The Essence of Linear Algebra"[1] for a really powerful visual and intuitive explanation. It starts with vectors and builds to matrix multiplication and further to several other topics. [1]: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...
  - bandrami 10y ago

lima 10y ago

Appropriate use of the .xyz TLD :-)

joshaidan 10y ago

Interesting, I wonder why Facebook has this website blocked, preventing me from posting it on my wall.

staltz 10y ago
Good sign that it's time to leave Facebook. Really
roywiggins 10y ago
Maybe the slightly unusual TLD?
geonnave 10y ago
I just had the same problem! I wanted to share it with the College group from where I graduated, but I could not do it.

mfirmin 10y ago

This reminds me of Nicky Case's Explorable Explanation of a 2D transformation matrix: http://ncase.me/matrix/ http://ncase.me/matrix/

wodenokoto 10y ago

As others have mentioned, many introductions to linear algebra / matrix multiplication only gives the student a formula, but not why the formula looks like it does. Why is matrix multiplication not pointwise, was something I had a hard time understanding. Better explained has a good article on this, that really made it click for me https://betterexplained.com/articles/linear-algebra-guide/ https://betterexplained.com/articles/linear-algebra-guide/

hacker_9 10y ago

As a gamedev who uses matrices all the time, I don't find this animation useful at all for practical work. Instead I think of matrix multiplication as creating a list of ordered operations (scale x translate x rotate etc) which is just encoded efficiently in a table, to then be sent to other parts of the program for use. You can even remove items from the list by multiplying by their inverse.

scraft 10y ago
But of course when you have encoded the scale, rotate and translate into the matrix, it isn't possible to decode it back out to your original values. This is obvious if you consider the fact the following produce identical matrices (just consider 2D for now): 1. Scale in X and Y of -1 2. Z rotation of 180
- hacker_9 10y ago
  Yes, I know, what is your point exactly? When I say 'list of ordered operations' I mean that if I were to apply the matrix to any object, you can imagine each transform in the list being applied one after another to the object visually, resulting in the final position.

FeepingCreature 10y ago

This is sounding increasingly like the Monad situation, where everybody has their own incompatible approach for explaining it, and none of them make any sense. I feel like I understand Matrices _less_ now than when I started reading around here.

maxiepoo 10y ago
If by "Monad situation" you mean trying to learn a concept by just reading a single blog post then sure. Trying to learn something by scattered blog posts without engaging in material is a bad idea no matter what the problem is. For some reason nobody expects to read a blog post and understand the fundamental theorem of calculus without first learning intuition and doing calculations with derivatives and integrals. On the other hand, if you have been working through a concept and you read a blog post that gives a different perspective than you're used to, it might help. If you want to actually learn something, then get something like a book or series of posts and work through examples and explore the ideas yourself. But having more than one intuition is useful and complaining about the fact that there is more than one way to understand something is lazy.

thanatropism 10y ago

See the very first video in Strang's MIT Linear Algebra course on YouTube. There's two ways of seeing matrix multiplication (from a matrix POV, rather than as linear transformations).

adamnemecek 10y ago

I have found the Stanford EE263 class to be possibly the best class on linear algebra. This might be due to the fact that it's technically a class about linear dynamic systems not linear algebra per se http://ee263.stanford.edu/ http://ee263.stanford.edu/

user1713952 10y ago

The idea is good but I am afraid it misses the point entirely. Multiplying both columns of the right hand side matrix to the left hand side matrix is really not helping the target audience understand what is going on. 2 operations are going on as the same time. I believe https://www.youtube.com/watch?v=XkY2DOUCWMU https://www.youtube.com/watch?v=XkY2DOUCWMU is a much better way to illustrate the matrix multiplication (and provide a clear explanation of the why) .

ahane 10y ago
Beat me to it :) amazing channel!
staltz 10y ago
Comments like these (aside from providing a nice link) just make authors of content sad. Why say such thing? You can say "here is another helpful link", but "misses the point entirely"? The point is to provide a self-evident mnemonic, as explained here: https://github.com/staltz/matrixmultiplication.xyz https://github.com/staltz/matrixmultiplication.xyz Not nice.
- user1713952 10y ago
  Mr Staltz, please allow me to elaborate. I did not mean to be mean at all, nor to reduce the value of your work, excuse me if I made you feel like that. I am actually grateful that you made this animation as it allows one to reflect on the concept. If your kid proudly brings home a drawing, you are most probably going to greet him with warm encouragements no matter the quality of the art. The kid is in a process of developing his creative sense, learning, and forming his/her sense of reality. Comparing the drawing to what mature artists produce would certainly have a negative effect on the kid. The point is to provide a feedback relevant to the recipient of the critic. The same goes here with this animation. I respect your work but obviously disagree with the usefulness of it. You made an animation to explain the matrix product, which is a really good idea. Alas, I think the realization of it gives a confused picture of what it is. I doubt that a student with no prior understanding of matrix products will get much enlightment. Too many operations are going on at the same time. This means that it misses its intended audience: students in need of building an intuitive understanding of the matrix product. That's what all those javascript (among others) animations (ex: http://jackschaedler.github.io/circles-sines-signals/dft_introduction.html http://jackschaedler.github.io/circles-sines-signals/dft_int...) are all about: make it simpler to grasp a concept. Unfortunately, though beautiful, the animation is not clear. At best it provides a visual mnemonic for multiplying matrices which maybe useful for some but not for the younger version of me in need of understanding :) I realize that instead of providing such a harsh critic, I would have been better inspired to suggest improvements. Mea culpa :)

coin 10y ago

Uggg, why is zoom disable on mobile devices?

Vinkekatten 10y ago

Meh, the site is blocked at my workplace. Websense is really crummy.

esalman 10y ago

Another one for matrix inversion would be nice.

ww2 10y ago

I think this is overdone and does not help the intuition. two things: (1). the first step of transposing the second matrix 90 degree is unnecessary and confusing. A row vector and a column vector are different. This step confuses the two. (2). All the final elements are independent and should be calculated in parallel, the stepped anime is not necessary and does not reveal this basic fact.

inglor 10y ago

Question for staltz - are you happy with the cycle code at https://github.com/staltz/matrixmultiplication.xyz/ https://github.com/staltz/matrixmultiplication.xyz/ ? It feels like a lot of work for this sort of task which I feel could have been achieved by using an existing (mutable) matrix lib and drawing ad-hoc. It feels like an awful lot of code written - especially with all the `.something === "SOME_STRING_CONSTANT"` (couldn't those be typed directly in TS btw?). Still, you're a smart guy - I'm trying to see why it's worth it. Do you really feel model-view-intent shines here? (Nice visualization by the way)

staltz 10y ago
Yes, I'm very happy with that code, apart from one thing: how animations are done sometimes with tweens in JS, and sometimes with CSS. I'm sure it could have been built with less code. In general, the more shortcuts you take and the less abstraction you use, the less amount of code you will have. But also, it's likely that you would mix concerns more easily. So in general, when separating concerns, you end up getting more LOCs. That's what I've perceived in my experience, not a fact that I can prove. So I do feel like model-view-intent shines here. Model only has state concerns. I was careful in moving away DOM-related code from the model. So theoretically you could reuse this model for React Native or any other UI target. View only displays state in a visual representation, and intent only captures user events and represents them in a form suitable for the model. About some TypeScript code, I think you referred to https://github.com/staltz/matrixmultiplication.xyz/blob/master/src/Calculator/model/actions.ts#L58-L80 https://github.com/staltz/matrixmultiplication.xyz/blob/mast.... These could have been avoided. The only purpose they serve is to make inference work out for lines like these: https://github.com/staltz/matrixmultiplication.xyz/blob/master/src/Calculator/model/reducers/controlPanel.ts#L16-L17 https://github.com/staltz/matrixmultiplication.xyz/blob/mast... TypeScript knows that `action` is of type StartMultiplyAction. If we had only done `.filter(ac => ac.type === 'START_MULTIPLY')`, TypeScript couldn't infer anymore. So I understand those isFoo functions are very boilerplatey, but they serve the purpose to provide accurate typings. It's optional and depending how you write TypeScript code, you can use more type casting and more `any`, or you can go down the route of providing accurate types everywhere, which is what I did. This is not at all related to Cycle.js nor MVI. Feel free to ask more architecture-related questions. I like these. What else feels like "too much code"?

hdivider 10y ago

Awesome. Some concepts in math are so much easier to grasp with an interactive visual. I once made an entire game around matrix multiplication -- such that players didn't even know they learned how to do this. And sometimes, visualizations can help cause discoveries. A lot of progress in theoretical physics came about because physicists 'guessed' what nature's equations should look like...and then got shocked when they discovered they were right. Take James Clerk Maxwell, for instance: "He had achieved his greatest success by relying on just one empirical fact known to every school boy. Yet now he tried to do without any empirical facts, by pure thinking. He believed in the power of reason to guess the laws according to which God has built the world." http://www.mathpages.com/home/kmath103/kmath103.htm http://www.mathpages.com/home/kmath103/kmath103.htm

abecedarius 10y ago
I'm curious about this game. Is it online?
- hdivider 10y ago
  I built it for the Windows 8 Store (of all places), years ago, in base C++/Cx. All de-published now though. Could dig up a gameplay video if you wish...
  - abecedarius 10y ago
    Thanks, but I'm just curious and don't like videos. The world needs more things like that, though -- it was nice to hear of it.

amelius 10y ago

How I would explain it: - First, show how to multiply a row vector by a column vector. - Then, show how to "simultaneously" multiply several row vectors by a single column vector by placing the row vectors in a matrix. - Now, similarly, do the same with the column vector. And voila, matrix by matrix multiplication.

appleflaxen 10y ago
im trying to understand this, and what you said appeals to me but i can't quite visualize what you mean. do you have an illustration?

jakehoon 10y ago

This would have been so awesome to have in college. I stumbled upon RegExr the other day (http://regexr.com http://regexr.com) which I absolutely love. It kind of shares this spirit of showing people how to do things that appear complex at first glance, but have a sort of addicting nuance to them. Thanks for sharing.

rsp1984 10y ago

Very nice, but the mental model I prefer is to think of a Matrix-Vector multiplication as the weighted sum of Matrix columns, and a Matrix-Matrix multiplication as just several Matrix-Vector multiplications (results being the columns of a new matrix).

tskaiser 10y ago

This is cool, but I have a few criticisms: - Flipping something over like that looks cool, but confuses the brain. At least it confused mine. - Inconsistently calculating 1 or 2 elements at a time is confusing. Having 2 results calculated at a time simply put requires the recipient to untangle what is going on. This only gets worse when the recipient tries to do it themselves for examples with more columns in the right matrix. Disclaimer: I am biased as I already know how to do matrix operations, and I visualize it differently. When multiplying by hand I translate the right matrix up, like in the animation, but do not flip it over. Then I calculate each element one at a time by taking the sum of products of the row and column that intersects over that element. Like so: http://i.imgur.com/D572zOI.png http://i.imgur.com/D572zOI.png As you can see I'm not much of a graphics artist. Edit: actually, I think this would be really good if you "filter down" one column at a time, instead of moving the whole matrix down, so that it only calculates one element at a time, one column at a time. Means the animation will contain more steps, but I think that is actually to its benefit. Does this make sense?

qb45 10y ago
> Flipping something over like that looks cool, but confuses the brain. At least it confused mine. I think it would be helpful to have a shadow of the right matrix after it's moved to the top. This way it would be easier to see where particular numbers are coming from in the original input. Disclaimer: I'm similarly biased and actually the reason I came up with this shadow thing is because I also imagine the right matrix moving like here but somehow this never causes it to disappear from its original place on paper, go figure, so this animation looked really alien to me.

ryanmonroe 10y ago

For me personally, a simple animated version of this image[0] looping through rows of A and columns of B seems easier to understand. [0] https://upload.wikimedia.org/wikipedia/commons/e/eb/Matrix_multiplication_diagram_2.svg https://upload.wikimedia.org/wikipedia/commons/e/eb/Matrix_m...

j1vms 10y ago

Not sure if anyone here has mentioned this yet, but it's useful to think of matrix multiplication as the most natural extension of one-dimensional algebra into multiple dimensions. First, remember that people first wrestled with and found solutions to "1-D" equations like "ax^2 + bx + c = 0". When others came along asking about how to extend these results into multiple dimensions so that previous results might still be useful, the most natural answer was found to be how we now define matrix multiplication. To see further evidence of this, take your favorite linear algebraic equation using matrices and then look at the edge case where you have only 1x1 matrices in your equation. Usually you'll end up with something that looks analogous to a result in basic algebra in real number (or complex) variables, not matrices. If all this still seems counter-intuitive, then realize this is essentially why math is so important to our logic. It helps us to reason about things that are not obvious, yet logically must fall through from the basic tautologies we "chose" (like 0 != 1, addition on integer numbers, etc.). Or, more aptly in this case, it helps retain an analogous symbolic representation (the overall logic of the equation at the high level) despite a change in the underlying nuts and bolts of computing solutions to it (matrix multiplication vs. our regular "1D" multiplication).

gunnihinn 10y ago
I'm sorry, but that's not how that works. Your example is of a second-degree equation, which can't be solved by linear algebra. Linear algebra was invented to solve systems of linear equations, and matrices first arose as a convenient notation for expressing those. Matrix multiplication came later and is now understood to correspond to the composition of linear morphisms on vector spaces. If you try to extend the (very few) methods we have for obtaining exact solutions to polynomial equations in one variable to higher dimension, you (a) fail and (b) invent algebraic geometry.

jayajay 10y ago

It's easier to think of matrix multiplication by computing the matrix elements, which means reducing the problem to NM vector dot products. To get the element of the result matrix at position (n,m), compute the inner product < r_n | c_m >, where r_n is the nth row vector of the left matrix and c_m is the mth column vector of the right matrix. Once you try it, you'll see that it's also much easier to visualize than this strange, unintuitive approach. With great power comes great responsibility. That domain name belongs to that person. I hope they do society a favor and put more content regarding MM that that, because they chose one of the least intuitive ways of thinking about it. Put more ways up, or ditch the domain. If the goal is to get people to understand what matrices are, the best way is to teach them about operators, transformations, vector spaces and linear algebra in general, because this is really the only way to fully understand what's going on without relying on some heuristic. If the goal is to get people to remember how to do matrix multiplication, at least put up more ways of doing it.

staltz 10y ago
Nope, won't ditch the domain. I'd recommend you ditch that type of speech.
- deleted 10y ago
  [deleted]
- 0xFFFFFF 10y ago
  Everyone's a critic, don't let it get to you.
- jayajay 10y ago
  I'm sorry if my words upset you. That said, it would be a shame to tell someone off if they bring up a good point (however brash). I used to teach this stuff, and I can tell you many of my students would end up being more confused with your visualization. I hope you can see that you've assumed some responsibility. That's all I can hope.

cocktailpeanuts 10y ago

I have no idea what's going on there. But it's a cool piece of art, I'm guessing that was the intention.

jordigh 10y ago

I find this image a lot easier to work with. The animation above just seems a tad confusing: https://staff.fnwi.uva.nl/r.vandenboomgaard/IPCV/_images/Matrix_multiplication_diagram_2.png https://staff.fnwi.uva.nl/r.vandenboomgaard/IPCV/_images/Mat...

tvural 10y ago

Matrices are linear maps, and matrix multiplication composes the linear maps. Now statements like the determinant of the product is the product of the determinants, the trace is preserved under similarity transforms (since similarity transforms just rewrite the linear map in a different basis), etc. are intuitive. I've always felt that these explicit calculations don't really get to the point. You can memorize them and still not really understand what's going on.

HQvIdsGkVw 10y ago
When I was a math student, we had two semesters of linear algebra. We only saw matrices at the end of the first semester; other than that it was all about linear mappings. The thing is, most linear algebra classes that are taught to other sections (biology, CS, …) start directly with matrices and are mostly computational.
gunnihinn 10y ago
Maybe you know something I don't, but I don't find these statements about the determinant and trace intuitive just from knowing that matrix multiplication corresponds to composition of linear maps. They can be made intuitive by appealing to geometry, but I don't see how to do that just by staring at the algebraic picture. Actually, proving them without picking a basis and then showing that what you get is invariant under base change is quite nontrivial and involves throwing a lot of heavy machinery around; see for example Coffman's non-coordinate proof of the statement about the trace in http://users.ipfw.edu/CoffmanA/pdf/book.pdf http://users.ipfw.edu/CoffmanA/pdf/book.pdf For the determinant, if you want to do everything without picking a basis, you're basically proving that the exterior power operation defines a functor on the category of finite-dimensional vector spaces, which isn't that bad to do, but you somehow have to explain what all those words mean along the way.

scalablenotions 10y ago

Thank you! This is marvelous and really made me understand this process at a much simpler level. Please consider making more for other mathematical concepts :)

HeavyStorm 10y ago

I read the title and thought, oh God, not another one, the second was bad enough and I like to pretend that the third never happened...

matteuan 10y ago

Nice, now let's do it with the Strassen's algorithm

JabavuAdams 10y ago

Neat visualization. I feel it's important and eye-opening to note that there's not just one way to multiply matrices, there're five (according to Strang): 5 ways to multiply A B = C 1) Each element of C is a dot-product of corresponding rows of A with columns of B 2) Each column of C is a combination of the columns A. Each column of B has the coefficients. 3) Each row of C is a combination of the rows of B. Each row of A has the coefficients. 4) Accumulate outer-product. Accumulate sums of of outer-products of each col of A with each row of B. 5) Block multiplication They all produce the same answer, but different ones provide different insight for particular situations. See Video Lecture #3 of Strang's Linear Algebra course for more info: https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/lecture-3-multiplication-and-inverse-matrices/ https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra...

jasonx1e 10y ago
His lectures are awesome, wish some of them could be remastered though

mrcactu5 10y ago

this is more than just visualization. i can touch the computation. each step is tangible

jasonx1e 10y ago

Now make one that can find inverses ( ͡° ͜ʖ ͡°)

kafkaesq 10y ago

I my head I see the matrices remaining in place, while the vectors from the left slowly float to the right, produce dot products, which in turn float into their destination cells in the new output matrix just to the right of the equals sign. But that's just me.

greggman 10y ago

This is very cool. For me I needed to know why it works not what it does. I worked it out here for 2d transformations http://webglfundamentals.org/webgl/lessons/webgl-2d-matrices.html http://webglfundamentals.org/webgl/lessons/webgl-2d-matrices... and then expanded it to 3d, perspective, and cameras.

ivan_ah 10y ago

Nice illustration of the "row picture" of matrix multiplication. The "column picture" is equally important. This intro lecture by Prof. Strang is a really good lesson on that: https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/lecture-1-the-geometry-of-linear-equations/ https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra... Also, this short tutorial on linear algebra covers the row + column pictures on the first page: https://minireference.com/static/tutorials/linear_algebra_in_4_pages.pdf https://minireference.com/static/tutorials/linear_algebra_in... (disclaimer: I'm the author)

vladsotirov 10y ago

The UI confuses me to the point of frustration. 1. A "Previous" button would help since otherwise you have to go loop through the whole thing to see what actually happened (e.g. how exactly the first step lifts and rotates the matrix, what's getting added to what). 2. The final animation step is a transition from the patently false equation "[matrix1]=[result_matrix]" to the finally correct equation "[matrix1] x [matrix2] = [result_matrix]". This could be avoided if instead of transforming "[matrix1] x [matrix2]" into "[matrix1 plus overlay]=[result_matrix to be filled]" and then doing the 'waterfall', the equation "[matrix1] x [matrix2] = [result_matrix to be filled]" was maintained throughout, perhaps with the moving overlay slightly grayer after lifting [matrix2].