4 ms·

> "For an avid student with great skill in mathematics, rushing through the standard curriculum is not the best answer. That student who breezed unchallenged th

> "For an avid student with great skill in mathematics, rushing through the standard curriculum is not the best answer. That student who breezed unchallenged through algebra, geometry, and trigonometry, will breeze through calculus, too." That was me. I was great at calculus type things, but Matrix Theory hit me like a ton of bricks. I still have that text book, sitting on my other desk, staring menacingly at me from across the room; Matrix Analysis, Horn and Johnson. Geometry in High School gave me a taste, but would have been nice had we had available another proof based class in the math curriculum; Formal Logic or Discrete Maths at a high school level. Maybe even Linear Algebra?

msla 8y ago
> That was me. I was great at calculus type things, but Matrix Theory hit me like a ton of bricks. I still have that text book, sitting on my other desk, staring menacingly at me from across the room; Matrix Analysis, Horn and Johnson. Geometry in High School gave me a taste, but would have been nice had we had available another proof based class in the math curriculum; Formal Logic or Discrete Maths at a high school level. Maybe even Linear Algebra? I am deeply confused by a curriculum which separates Matrix Theory from Linear Algebra. The description in the Wikipedia category just barely helps: https://en.wikipedia.org/wiki/Category:Matrix_theory https://en.wikipedia.org/wiki/Category:Matrix_theory > Matrix theory is a branch of mathematics which is focused on study of matrices. Initially, it was a sub-branch of linear algebra, but soon it grew to cover subjects related to graph theory, algebra, combinatorics and statistics as well. The University of Missouri has a Matrix Theory course: https://www.math.missouri.edu/class/matrix-theory https://www.math.missouri.edu/class/matrix-theory > Basic properties of matrices, determinants, vector spaces, linear transformations, eigenvalues, eigenvectors, and Jordan normal forms. Introduction to writing proofs. ... which specifies a textbook: > Linear Algebra with Applications (7th edition) by Steven J. Leon ... which deepens my confusion. If you're taking that course, how is it not an introductory Linear Algebra course? And this MathOverflow answer obfuscates again: https://mathoverflow.net/questions/11669/what-is-the-difference-between-matrix-theory-and-linear-algebra https://mathoverflow.net/questions/11669/what-is-the-differe... > Let me elaborate a little on what Steve Huntsman is talking about. A matrix is just a list of numbers, and you're allowed to add and multiply matrices by combining those numbers in a certain way. When you talk about matrices, you're allowed to talk about things like the entry in the 3rd row and 4th column, and so forth. In this setting, matrices are useful for representing things like transition probabilities in a Markov chain, where each entry indicates the probability of transitioning from one state to another. You can do lots of interesting numerical things with matrices, and these interesting numerical things are very important because matrices show up a lot in engineering and the sciences. > In linear algebra, however, you instead talk about linear transformations, which are not (I cannot emphasize this enough) a list of numbers, although sometimes it is convenient to use a particular matrix to write down a linear transformation. The difference between a linear transformation and a matrix is not easy to grasp the first time you see it, and most people would be fine with conflating the two points of view. However, when you're given a linear transformation, you're not allowed to ask for things like the entry in its 3rd row and 4th column because questions like these depend on a choice of basis. Instead, you're only allowed to ask for things that don't depend on the basis, such as the rank, the trace, the determinant, or the set of eigenvalues. This point of view may seem unnecessarily restrictive, but it is fundamental to a deeper understanding of pure mathematics. If I try to parse charitably, I come away with the idea that Matrix Theory is about matrices as a data structure, usable for many things outside the scope of Linear Algebra, where they're all about using matrices to represent linear transformations. It's the difference between a column of numbers on a shopping bill and a column of numbers which represents a vector in a space with a specified basis. Gotcha. However, this answer directly contradicts what the University of Missouri calls Matrix Theory, which is so Linear Algebra they even use a Linear Algebra textbook. It also... I don't know, trivializes the field of Matrix Theory. So you can manipulate matrices. So what? They show up a lot because they're used to represent specific things. Is the course going to barely introduce a lot of specific things and then focus on the matrix representation? What a waste!
- RandomInteger4 8y ago
  I didn't say that Linear Algebra and Matrix Theory were separated. I said that Matrix theory hit me like a ton of bricks. I took Linear Algebra in college prior to that, obviously. I further stated that I think Linear algebra might benefit students if taught earlier, in high school.
  - Gibbon1 8y ago
    We were taught a little linear algebra in high school. And more in college. My impression of three semesters of calculus in college was that much was a waste of time. It was probably useful for a mechanical/electrical engineer circa 1950. But today no one solves problems that way. I think more linear algebra and matrix theory would have been better.
- siddboots 8y ago
  I think it will serve better to think about it as a difference of emphasis, rather than a difference of category. Maybe the term has been introduced to make the subject seem more tangible to newcomers? I notice that University of Missouri doesn't also have a "linear algebra" course. As you say, "Matrix Theory is about matrices as a data structure" ends up being a pretty hollow concept anyway, because the important part is always related to matrix multiplication. The example of Markov chain models only underlines that point, since the main results of that theory depend entirely on the transition function being linear, and not at all on whether we represent it by a matrix. To put it another way: matrix multiplication is composition of linear operators. You can't extricate the matrix-as-data concept from the other.
- jacobolus 8y ago
  Your interpretation of “matrix theory” doesn’t seem to have much to do with the course mentioned by RandomInteger4. You can see what the content was by looking at the textbook mentioned, https://amzn.com/0521548233 https://amzn.com/0521548233 Judging from the reviews it seems like it is a good reference book of intermediate/advanced linear algebra topics which researchers in other fields found useful as a reference. I’m guessing this course was intended as maybe a 3rd course in linear algebra, with a slightly applied flavor. Giving it a different name makes it easier for students to distinguish the course than just calling it “Linear algebra 3A” or whatever.
graycat 8y ago
I'll try to clear things up: First, Horn and Johnson is a bit much. I was in Horn's class. I had done a LOT in, call it, linear algebra and matrix theory in my career before the class, told the profs I didn't need the course, and they said it was a "second, advanced course" and smiled. The course was quite competitive and without trying at all and without intending to be competitive, I effortlessly blew away all the other students on graded homework, the tests, the midterm, the final exam, and the corresponding qualifying exam. At the end of the course Horn wrote about me IIRC "Best performance in the class by a wide margin. Knows this material cold." So, yes, it was an advanced course, actually had a lot of nice stuff in it, Horn's lectures were nicely precise and at times with some unusual, nice approaches, but to do well in the course it was sufficient just to have had a good background before. What background? For the main books, E. Nering (a student of E. Artin at Princeton), Halmos (an assistant to von Neumann at the Institute of Advanced Study at Princeton), Finite Dimensional Vector Spaces, basically also a finite introduction to Hilbert space and the spectral theorem there, Forsythe and Moler, Computer Solutions of Linear Algebraic Systems, and some good texts in multivariate statistics with regression analysis, discriminate analysis, factor analysis, analysis of variance. More in applications, e.g., the fast Fourier transform, more on curve fitting, linear systems in electronic engineering, antenna theory and beam forming, optimization, linear programming, unconstrained optimization, the Markowitz and Sharpe applications to finance, Lagrange multipliers, the Kuhn-Tucker conditions, etc. can also help. But Horn is not a good choice for a first text. For a first or second text I'd suggest, say, Hoffman and Kunze, Linear Algebra, Second Edition available for free on the Internet. For more, see my post on math in https://news.ycombinator.com/item?id=15116379 https://news.ycombinator.com/item?id=15116379 and there sections (2) Linear Algebra (2.1) Linear Equations (2.2) Gauss Elimination (2.3) Vectors and Matrices (2.4) Vector Spaces (2.5) Eigen Values, Vectors (2.6) Texts To be brief, about the earliest and easiest start on linear algebra and matrix theory is just a high school style system of linear equations. The main solution technique is Gauss elimination. Matrix notation is a better notation for that subject. Here is essentially the role of matrix theory: Each of the old results in linear algebra can be written as a result, with nicer notation, in matrix theory. Can get the same results without matrix notation, but matrix notation makes it all much easier. Next, a broad statement is that the two pillars of the field of analysis in math are (1) linearity and (2) continuity. Well, linear algebra and matrix theory stands strongly on linearity and, as we move on in both the theory and applications, also continuity. Let's be clear on linearity via linear algebra and matrix theory: So, for positive integers m and n and an m x n matrix A we say that matrix A is a linear transformation (function) if for all n x 1 vectors x and y, and numbers a and b, we have that A(ax + by) = aAx + bAy Sure, to read this need the definitions of matrix sum and product; sum is trivial; product is not much harder and is really just what need to make Ax = b be the same as the high school system of linear equations. For the numbers, usually use either the set of real numbers R or the set of complex numbers C. But, sure, for numerical computation are essentially limited to the set of rational numbers Q. But in general need only what a course in abstract algebra calls a field: Each of R, C, and Q is such a field but also the set of integers modulo a prime number is a field, of interest in algebraic coding theory and cryptology. This definition of linearity generalizes in Hilbert space, Banach space, and functional analysis, and the more general definitions and results are important in quantum mechanics, differential equations in science and engineering, signal processing in electronic engineering, etc. Again, linearity is a pillar of analysis in math. Why pillars? In both theory and applications, linearity and continuity commonly hold and are astoundingly powerful properties. For such applications we have multivariate statistics, optimization, electronic engineering, antenna theory, beam forming, signals (each time invariant linear system has sines and cosines as eigenvectors; when a violinist on a concert stage plays some pure tones, the concert hall transmits those tones to you in the audience as a linear system so that what you hear are just the pure tones with the right frequencies but with some phase and amplitude changes; the Navy likes to know that for sonar signals; the USAF likes to know that for radar signals; cell phone people like to know that for their signals), Fourier theory, linear partial differential equations, superposition in quantum mechanics, etc. And when linearity does not hold, commonly it is a good, first approximation and the main means of iterative techniques. And if a problem is not linear, maybe after some simple transformation it will be. In some of the posts here, there is mention of matrix theory and basis, that is, a coordinate system. Well, can do that although is it not nearly as general as what physics likes to do with coordinate systems. But also can just decide not to do that, to take the vector space as just the n-tuples and not force thinking of the n-tuples as just coordinates of vectors in some basis. Or can do either approach depending on what is easier in the context. Here is a point should get: Suppose we start with just systems of linear equations. Then we say we are working with n-tuples of numbers. Then we call those n-tuples a vector space. Then using essentially just the main, relevant properties of those n-tuples, we write down the definition, axioms, of a vector space where we've said nothing about the vectors but have left them as just points. Well then we have two advantages: First, the definition of a vector space lets us talk about subspaces and in particular subspaces of the vector spaces of just the n-tuples, and we want to do that already, strongly with just Gauss elimination for linear equations. E.g., with the linear system Ax = b where m, n are positive integers, A is m x n, x is n x 1, and b is m x 1, the set of all x so that Ax = 0 (m x 1 of all zeros) is a vector subspace of all the n x 1 vectors (for the set of real numbers R, commonly called the set R^n). Call the set K the set of all x so that Ax = 0. If for some n x 1 u we have that Au = b, then from linearity we can argue that any v so that Av = b can be written as a sum of u and some vector in K. In this way we see all possible solutions of Ax = b. Actually at the end of Gauss elimination we can see K and u easily enough. Second, we get to consider vectors other than just n-tuples. E.g., we can consider the data of 1 second of music as a vector, a random variable as a vector, a color as a vector, the wave function of a photon or electron as a vector, etc. Then as the book continues, we get into eigenvalues and eigenvectors. Eigen is German essentially for special. They are special, and valuable. Maybe the nicest part is the polar decomposition: Each square matrix is a product UH where U in unitary and H is Hermitian. In class, when Horn got to that, I shouted out "That's my favorite theorem! The unitary part is an isometry, that is, doesn't change lengths or angles and is essentially a rigid motion, maybe just a rotation or reflection. The Hermitian part H is a shocking dream, amazing beyond belief: All H can do is take a circle and make it into an ellipse: The two axes of the ellipse are perpendicular (orthogonal) and the eigenvectors. Their lengths are the eigenvalues. And this generalizes to rounded footballs in three dimensions and all finite dimensions. And with the spectral theorem it generalizes to infinitely many dimensions and is the main reason in quantum mechanics the observables are eigenvalues. The polar decomposition is also the source of the powerful singular value decomposition, principle components analysis, factor analysis, analysis of saddle points in optimization (see W. Fleming, Functions of Several Variables), the matrix condition number in the numerical analysis of Gauss elimination, and much more in theory and applications. Hope this helps.
tomrod 8y ago
I love that book! It's certainly not an entry point though.

msla 8y ago

> That was me. I was great at calculus type things, but Matrix Theory hit me like a ton of bricks. I still have that text book, sitting on my other desk, staring menacingly at me from across the room; Matrix Analysis, Horn and Johnson. Geometry in High School gave me a taste, but would have been nice had we had available another proof based class in the math curriculum; Formal Logic or Discrete Maths at a high school level. Maybe even Linear Algebra? I am deeply confused by a curriculum which separates Matrix Theory from Linear Algebra. The description in the Wikipedia category just barely helps: https://en.wikipedia.org/wiki/Category:Matrix_theory https://en.wikipedia.org/wiki/Category:Matrix_theory > Matrix theory is a branch of mathematics which is focused on study of matrices. Initially, it was a sub-branch of linear algebra, but soon it grew to cover subjects related to graph theory, algebra, combinatorics and statistics as well. The University of Missouri has a Matrix Theory course: https://www.math.missouri.edu/class/matrix-theory https://www.math.missouri.edu/class/matrix-theory > Basic properties of matrices, determinants, vector spaces, linear transformations, eigenvalues, eigenvectors, and Jordan normal forms. Introduction to writing proofs. ... which specifies a textbook: > Linear Algebra with Applications (7th edition) by Steven J. Leon ... which deepens my confusion. If you're taking that course, how is it not an introductory Linear Algebra course? And this MathOverflow answer obfuscates again: https://mathoverflow.net/questions/11669/what-is-the-difference-between-matrix-theory-and-linear-algebra https://mathoverflow.net/questions/11669/what-is-the-differe... > Let me elaborate a little on what Steve Huntsman is talking about. A matrix is just a list of numbers, and you're allowed to add and multiply matrices by combining those numbers in a certain way. When you talk about matrices, you're allowed to talk about things like the entry in the 3rd row and 4th column, and so forth. In this setting, matrices are useful for representing things like transition probabilities in a Markov chain, where each entry indicates the probability of transitioning from one state to another. You can do lots of interesting numerical things with matrices, and these interesting numerical things are very important because matrices show up a lot in engineering and the sciences. > In linear algebra, however, you instead talk about linear transformations, which are not (I cannot emphasize this enough) a list of numbers, although sometimes it is convenient to use a particular matrix to write down a linear transformation. The difference between a linear transformation and a matrix is not easy to grasp the first time you see it, and most people would be fine with conflating the two points of view. However, when you're given a linear transformation, you're not allowed to ask for things like the entry in its 3rd row and 4th column because questions like these depend on a choice of basis. Instead, you're only allowed to ask for things that don't depend on the basis, such as the rank, the trace, the determinant, or the set of eigenvalues. This point of view may seem unnecessarily restrictive, but it is fundamental to a deeper understanding of pure mathematics. If I try to parse charitably, I come away with the idea that Matrix Theory is about matrices as a data structure, usable for many things outside the scope of Linear Algebra, where they're all about using matrices to represent linear transformations. It's the difference between a column of numbers on a shopping bill and a column of numbers which represents a vector in a space with a specified basis. Gotcha. However, this answer directly contradicts what the University of Missouri calls Matrix Theory, which is so Linear Algebra they even use a Linear Algebra textbook. It also... I don't know, trivializes the field of Matrix Theory. So you can manipulate matrices. So what? They show up a lot because they're used to represent specific things. Is the course going to barely introduce a lot of specific things and then focus on the matrix representation? What a waste!

RandomInteger4 8y ago
I didn't say that Linear Algebra and Matrix Theory were separated. I said that Matrix theory hit me like a ton of bricks. I took Linear Algebra in college prior to that, obviously. I further stated that I think Linear algebra might benefit students if taught earlier, in high school.
- Gibbon1 8y ago
  We were taught a little linear algebra in high school. And more in college. My impression of three semesters of calculus in college was that much was a waste of time. It was probably useful for a mechanical/electrical engineer circa 1950. But today no one solves problems that way. I think more linear algebra and matrix theory would have been better.
siddboots 8y ago
I think it will serve better to think about it as a difference of emphasis, rather than a difference of category. Maybe the term has been introduced to make the subject seem more tangible to newcomers? I notice that University of Missouri doesn't also have a "linear algebra" course. As you say, "Matrix Theory is about matrices as a data structure" ends up being a pretty hollow concept anyway, because the important part is always related to matrix multiplication. The example of Markov chain models only underlines that point, since the main results of that theory depend entirely on the transition function being linear, and not at all on whether we represent it by a matrix. To put it another way: matrix multiplication is composition of linear operators. You can't extricate the matrix-as-data concept from the other.
jacobolus 8y ago
Your interpretation of “matrix theory” doesn’t seem to have much to do with the course mentioned by RandomInteger4. You can see what the content was by looking at the textbook mentioned, https://amzn.com/0521548233 https://amzn.com/0521548233 Judging from the reviews it seems like it is a good reference book of intermediate/advanced linear algebra topics which researchers in other fields found useful as a reference. I’m guessing this course was intended as maybe a 3rd course in linear algebra, with a slightly applied flavor. Giving it a different name makes it easier for students to distinguish the course than just calling it “Linear algebra 3A” or whatever.

graycat 8y ago

I'll try to clear things up: First, Horn and Johnson is a bit much. I was in Horn's class. I had done a LOT in, call it, linear algebra and matrix theory in my career before the class, told the profs I didn't need the course, and they said it was a "second, advanced course" and smiled. The course was quite competitive and without trying at all and without intending to be competitive, I effortlessly blew away all the other students on graded homework, the tests, the midterm, the final exam, and the corresponding qualifying exam. At the end of the course Horn wrote about me IIRC "Best performance in the class by a wide margin. Knows this material cold." So, yes, it was an advanced course, actually had a lot of nice stuff in it, Horn's lectures were nicely precise and at times with some unusual, nice approaches, but to do well in the course it was sufficient just to have had a good background before. What background? For the main books, E. Nering (a student of E. Artin at Princeton), Halmos (an assistant to von Neumann at the Institute of Advanced Study at Princeton), Finite Dimensional Vector Spaces, basically also a finite introduction to Hilbert space and the spectral theorem there, Forsythe and Moler, Computer Solutions of Linear Algebraic Systems, and some good texts in multivariate statistics with regression analysis, discriminate analysis, factor analysis, analysis of variance. More in applications, e.g., the fast Fourier transform, more on curve fitting, linear systems in electronic engineering, antenna theory and beam forming, optimization, linear programming, unconstrained optimization, the Markowitz and Sharpe applications to finance, Lagrange multipliers, the Kuhn-Tucker conditions, etc. can also help. But Horn is not a good choice for a first text. For a first or second text I'd suggest, say, Hoffman and Kunze, Linear Algebra, Second Edition available for free on the Internet. For more, see my post on math in https://news.ycombinator.com/item?id=15116379 https://news.ycombinator.com/item?id=15116379 and there sections (2) Linear Algebra (2.1) Linear Equations (2.2) Gauss Elimination (2.3) Vectors and Matrices (2.4) Vector Spaces (2.5) Eigen Values, Vectors (2.6) Texts To be brief, about the earliest and easiest start on linear algebra and matrix theory is just a high school style system of linear equations. The main solution technique is Gauss elimination. Matrix notation is a better notation for that subject. Here is essentially the role of matrix theory: Each of the old results in linear algebra can be written as a result, with nicer notation, in matrix theory. Can get the same results without matrix notation, but matrix notation makes it all much easier. Next, a broad statement is that the two pillars of the field of analysis in math are (1) linearity and (2) continuity. Well, linear algebra and matrix theory stands strongly on linearity and, as we move on in both the theory and applications, also continuity. Let's be clear on linearity via linear algebra and matrix theory: So, for positive integers m and n and an m x n matrix A we say that matrix A is a linear transformation (function) if for all n x 1 vectors x and y, and numbers a and b, we have that A(ax + by) = aAx + bAy Sure, to read this need the definitions of matrix sum and product; sum is trivial; product is not much harder and is really just what need to make Ax = b be the same as the high school system of linear equations. For the numbers, usually use either the set of real numbers R or the set of complex numbers C. But, sure, for numerical computation are essentially limited to the set of rational numbers Q. But in general need only what a course in abstract algebra calls a field: Each of R, C, and Q is such a field but also the set of integers modulo a prime number is a field, of interest in algebraic coding theory and cryptology. This definition of linearity generalizes in Hilbert space, Banach space, and functional analysis, and the more general definitions and results are important in quantum mechanics, differential equations in science and engineering, signal processing in electronic engineering, etc. Again, linearity is a pillar of analysis in math. Why pillars? In both theory and applications, linearity and continuity commonly hold and are astoundingly powerful properties. For such applications we have multivariate statistics, optimization, electronic engineering, antenna theory, beam forming, signals (each time invariant linear system has sines and cosines as eigenvectors; when a violinist on a concert stage plays some pure tones, the concert hall transmits those tones to you in the audience as a linear system so that what you hear are just the pure tones with the right frequencies but with some phase and amplitude changes; the Navy likes to know that for sonar signals; the USAF likes to know that for radar signals; cell phone people like to know that for their signals), Fourier theory, linear partial differential equations, superposition in quantum mechanics, etc. And when linearity does not hold, commonly it is a good, first approximation and the main means of iterative techniques. And if a problem is not linear, maybe after some simple transformation it will be. In some of the posts here, there is mention of matrix theory and basis, that is, a coordinate system. Well, can do that although is it not nearly as general as what physics likes to do with coordinate systems. But also can just decide not to do that, to take the vector space as just the n-tuples and not force thinking of the n-tuples as just coordinates of vectors in some basis. Or can do either approach depending on what is easier in the context. Here is a point should get: Suppose we start with just systems of linear equations. Then we say we are working with n-tuples of numbers. Then we call those n-tuples a vector space. Then using essentially just the main, relevant properties of those n-tuples, we write down the definition, axioms, of a vector space where we've said nothing about the vectors but have left them as just points. Well then we have two advantages: First, the definition of a vector space lets us talk about subspaces and in particular subspaces of the vector spaces of just the n-tuples, and we want to do that already, strongly with just Gauss elimination for linear equations. E.g., with the linear system Ax = b where m, n are positive integers, A is m x n, x is n x 1, and b is m x 1, the set of all x so that Ax = 0 (m x 1 of all zeros) is a vector subspace of all the n x 1 vectors (for the set of real numbers R, commonly called the set R^n). Call the set K the set of all x so that Ax = 0. If for some n x 1 u we have that Au = b, then from linearity we can argue that any v so that Av = b can be written as a sum of u and some vector in K. In this way we see all possible solutions of Ax = b. Actually at the end of Gauss elimination we can see K and u easily enough. Second, we get to consider vectors other than just n-tuples. E.g., we can consider the data of 1 second of music as a vector, a random variable as a vector, a color as a vector, the wave function of a photon or electron as a vector, etc. Then as the book continues, we get into eigenvalues and eigenvectors. Eigen is German essentially for special. They are special, and valuable. Maybe the nicest part is the polar decomposition: Each square matrix is a product UH where U in unitary and H is Hermitian. In class, when Horn got to that, I shouted out "That's my favorite theorem! The unitary part is an isometry, that is, doesn't change lengths or angles and is essentially a rigid motion, maybe just a rotation or reflection. The Hermitian part H is a shocking dream, amazing beyond belief: All H can do is take a circle and make it into an ellipse: The two axes of the ellipse are perpendicular (orthogonal) and the eigenvectors. Their lengths are the eigenvalues. And this generalizes to rounded footballs in three dimensions and all finite dimensions. And with the spectral theorem it generalizes to infinitely many dimensions and is the main reason in quantum mechanics the observables are eigenvalues. The polar decomposition is also the source of the powerful singular value decomposition, principle components analysis, factor analysis, analysis of saddle points in optimization (see W. Fleming, Functions of Several Variables), the matrix condition number in the numerical analysis of Gauss elimination, and much more in theory and applications. Hope this helps.

tomrod 8y ago

I love that book! It's certainly not an entry point though.