9 ms·
Why is it important for a matrix to be square? (2018)
- ptero 6y agoTLDR: the author is asking an honest question, not trying to say that non square matrices have little use in linear algebra. As linear transformations of space into itself, a very frequent operation, are described by a square matrix those matrices do show up more frequently. But map X into Y of different dimensions and you get a non-square matrix
- nextos 6y agoI think the key is that linear algebra textbooks do not tell students straight away that matrices are encodings of linear maps. Hence a square matrix is an encoding of a map that preserves dimensions. Axler, and a few others, say this straight away in the intro. This makes things much simpler to understand instead of developing linear algebra historically from systems of linear equations.
- dnautics 6y agoI happened to be the other way, I learned first that matrices were linear transformations and, being thick-headed, it took me a really long time to understand that matrices could also be used as "dumb excel spreadsheets". there's a fantastic historiography of linear algebra and why we're stuck in this situation here: https://www.youtube.com/watch?v=C2RO34b_oPM https://www.youtube.com/watch?v=C2RO34b_oPM
- nextos 6y agoA worse situation is that of statistics, where many textbooks insist on using approximations to the normal which obscure concepts and are not needed in the age of computer-based inference [1]. Calculus is also a bit confusing as for historical reasons we use a lot of notation from Newton-Leibniz informal infinitesimal approach, but we try to teach Bolzano-Weirstrass formal analysis, ending up with a confusing potpurri. [1] https://escholarship.org/uc/item/6hb3k0nz https://escholarship.org/uc/item/6hb3k0nz
- dnautics 6y agoI didn't mind the issue in calculus. The first thing we did in analysis after being introduced the Bolzano weierstrass form was to prove that it was equivalent to epsilon delta proofs.
- thaumasiotes 6y ago> But map X into Y of different dimensions and you get a non-square matrix This is true, but it's also something that is arguably impossible on the one hand and "unwise" on the other. You can easily map a higher-dimensional space into a lower-dimensional space, but you will irretrievably lose a lot of information when you do so. And in the other direction, you can't actually map a lower-dimensional space into a higher-dimensional space with a matrix. The image of X can never have more dimensions than X does -- the choice to represent it with Y > X dimensions is just that, a representational choice. This idea is only meaningful in terms of the semantics behind the representation.
- afwaller 6y agoSometimes losing that information is very useful however, to arrive at a representation that has a specific meaning. For example, an x-ray computed tomography (CT) image volume in 3D may be projected into various 2D synthetic planar x-ray projections (digitally reconstructed radiographs). There are countless situations where projections are very useful.
- thaumasiotes 6y agoI don't disagree, but I think my comment helps to illustrate a reason why square matrices really are more special than other matrices.
- unsrsly 6y agoIn fact, the 3D CT image itself is reconstructed from projections! See https://en.wikipedia.org/wiki/Tomographic_reconstruction https://en.wikipedia.org/wiki/Tomographic_reconstruction
- thaumasiotes 6y agoYes, but this is a case where you have three dimensions in the input and also three dimensions in the output. (You're assembling the 3D image from different 2D slices with different depth coordinates.) The third dimension is discretized while the other two are continuous; the reconstruction consists of smoothing out the third dimension.
- layoutIfNeeded 6y agoBecause it’s hip to be square!
- solinent 6y agoAn extension of spectral decomposition on a square matrix is called the singular value decomposition where any matrix can be decomposed. The trick is to diagonalize the matrix A A^T and A^T A, in this manner we can find the left and right singular values and have an equation which is very similar to the spectral decomposition. It has a very wide range of applicibility--most of which is also highly valuable. There's also the Moore-Penrose pseudoinverse which also has plenty of applicaitons and works on non-square matrices.
- ivan_ah 6y agoYeah. I don't know why LA courses focus so much on eigenvalues, and don't talk much about SVD. It seems like an obvious next step once you know how to compute eigenvalues, but SVDs are always covered in passing and superficially (at least in courses I've taken). I find it fascinating that A = U∑V^T works on ANY matrix A, with U and V being made of orthogonal vectors (the left and right singular vectors of A)... it's like saying that all matrices are scalings and projections when viewed with appropriate bases for the input and output spaces (U and V). > we can find the left and right singular values Interestingly the (nonzero subset) of the left and right singular values happen to be the same... I don't have a useful intuition about how to explain this. Does anyone know why A A^T and A^T A have the same eigenvalues?
- nextos 6y agoSVDs are the entry point to topic models, e.g. NNMF. Super interesting and useful for modeling large datasets efficiently.
- OscarCunningham 6y ago> Yeah. I don't know why LA courses focus so much on eigenvalues, and don't talk much about SVD. It makes sense to talk about the eigenvalues of an operator on any vector space. But in order for singular values to make sense the vector space needs to have an inner product. So eigenvalues are more general because they make sense even when you haven't chosen an inner product on your vector space. From an HN perspective this kind of consideration is relevant when working on a data set in which there are some arbitrary units. You don't want your model predicting house prices to work differently depending on whether you choose to measure floor size in m^2 or ft^2.
- mumblemumble 6y agoIf you're looking for a more intuitive understanding of linear algebra, I highly recommend 3blue1brown's YouTube series, "Essence of Linear Algebra."
- Izkata 6y agoAbsolutely this. If I remember right, it's roughly 13 videos of 20-30 minutes each. By the end I had a better understanding of what linear algebra was for and (conceptually) how the math actually worked than I got from a full semester at college.
- prostodata 6y agoIn the context of computer science, matrices are used as a representation construct with different possible formal interpretations which frequently can be recognized depending on the supported operations. For example, some functions of NumPy assume that matrices represent elements of linear algebra while other functions treat a matrix as a collection of rows or columns. In the latter case, a square matrix is probably an exception because rows and columns have completely different semantics.
- Koshkin 6y agoThe most important example of a non-square matrix is a (column) vector.
- raverbashing 6y agoWell, because matrices are "weird" mathematical objects. They're a composition of vectors (which make more sense than matrices) but it seems the square matrices are where you want to end up and non-square ones are kinda like special cases. If you're solving a linear system of equations for example: you need a square matrix. If you don't either your system is underdefined or you have redundancies or contradictions into it. (And one could say algebra with numbers is a special case of a 1x1 matrix, but anyway...) So worry about square matrices, about 1xN and Nx1 matrices (which are actually vectores) and the weird shapes are weird shapes
- gspr 6y agoWhat? They're a convenient device for writing down a linear map between finite-dimensional vector spaces. Nothing more, nothing less. Your explanation is bound to confuse people far more than help, as it seems to mystify something that is very mundane.
- raverbashing 6y agoYes they're mundane, but the "linear map between different vector spaces" case is pretty much the only thing they can do. Think of it as type casting in programming languages, you only do it when you need, but the real work is done processing elements of the same type. If that wasn't the case, then non-square matrices would have all the "cool" properties of the square ones. Yes I might be picking at straws here, and I agree it might confuse people even more, but this weirdness of matrices hasn't escaped me since I've learned about it. And then when you get to vectors it all makes sense. Hum...
- gspr 6y ago> Yes they're mundane, but the "linear map between different vector spaces" case is pretty much the only thing they can do. Of course it's all they "can do". They're merely a way of writing down linear maps for a given choice of bases.
- 6y ago
- deleted 6y ago[deleted]
- Synaesthesia 6y agoAn mxn matrix can be thought of as a mapping or function from one subspace Rn and another Rm. If n=m and the matrix is square, it is possible to have an inverse mapping, (inverse matrix).
- Item_Boring 6y agoThe reason the matrix A has to be square for det(A-XI_n) is a) because we cannot subtract a matrix A from another matrix B if their dimensions aren’t the same and since XI_n is a nxn matrix A has to be an nxn matrix as well and b) because the determinant is only defined for square matrices.
- Ceezy 6y agoIt s not more important! And most of these theorems can be extended to non square matrix anyway. Projections are examples of Rn to Rm applications that are very importants. Coordinates or probability measures too. And it make little sense to try to reduce them to endomorphism. Matrix can represent applications but they are objects worth studying for what they are.
- anonytrary 6y agoI think the answer here is simple and not as profound as people might think. Square matrices are linear operators which map an input vector into a space with the same dimensionality. We can characterize a few interesting symmetries when a matrix happens to be square. As such, we have more rules and theorems around them, making them suitable for practical usage. For example, physics deeply utilizes Hermitian matrices, which are a special subclass of square matrices. Spectral theory is incredibly important in all sorts of applications.
- anonytrary 6y agoSquare matrices are also a natural description for graphs (e.g. adjacency matrices), which are fundamental objects for pretty much all CS applications across all layers of your tech stack. Square matrices are great for representing pair-wise interactions between nodes.
- akater 6y agoAnother evidence that linear algebra concepts are terribly confusing when layed out without geometric background. This had been criticised for decades. Competently and popularly presented criticism goes back to at least as far as 1957 (Artin's Geometric Algebra; see discussion of determinants somewhere near the beginning) but linear algebra is still often presented decoupled from geometry. I wonder though if there's purely algebraic approach to matrices that explains as much (or more) as geometric one. Maybe approaching algebra of matrices consistently as an example of category algebra could be illuminating.
- dnautics 6y agoI learned abstract linear algebra first, and didn't learn geometrical interpretations until I taught myself many years later so I could write an asteroids clone in svg. I don't think it was a pedagogically a problem, except I couldn't bring myself to care about matrices when I was learning them... It was very easy to take my abstract knowledge and apply it, and for me it might have been harder the other way around. In retrospect a hybrid applied/theoretical topic (like reed Solomon encoding and recovery) might have perked me up. But I might have also been a strange case.
- akater 6y agoIf by “abstract” linear algebra you mean “course that starts with the definition of vector space”, then it is geometric enough in the sense Artin talks about. It is pedagogically a problem for people who ask questions like the one in topic. But it can also be a problem when one encounters bilinear form and linear operator in practice but can't distingish between the two. I can't think of a specific example but I was asked once about some problem (in electrical engineering, iirc) where the source of confusion was this; some transformations of a (square) matrix were natural while others were not. Some people feel strongly about the topic—mostly those with “pure math” inclinations.
- zeroimpl 6y agoThe question as asked in the title is incorrect. It is not important for a matrix to be square. Matrices are defined to solve specific problems, and the specific problem is what determines the geometry, not some goal of having square matrices. However the person actually asked why it is important for matrices to be square for most of the theorems they are learning. The actual answers to this are interesting, but the presence of this question likely implies that they are being taught abstract theory first prior to building much intuition. I was taught this way as well in my advanced pure math course. It was all super abstract until I was studying for the final exam and then had this eureka moment where suddenly everything made sense (a matrix is just a numerical way of describing a linear transform! And computing eigenvectors is like factoring!). Sadly, this happened again the next term where we derived SVD - except this time it never made sense to me until a later on course where we needed to use SVD for some application.
- raincom 6y agoV.I. Arnold said: "The determinant of a matrix is an (oriented) volume of the parallelepiped whose edges are its columns. If the students are told this secret (which is carefully hidden in the purified algebraic education), then the whole theory of determinants becomes a clear chapter of the theory of poly-linear forms. If determinants are defined otherwise, then any sensible person will forever hate all the determinants, Jacobians and the implicit function theorem." From: https://www.uni-muenster.de/Physik.TP/~munsteg/arnold.html https://www.uni-muenster.de/Physik.TP/~munsteg/arnold.html