8 ms·
How to Conquer Tensorphobia
- mrcactu5 13y agoAny array in several indices is using a tensor product. x[1][2][3] On StackOverflow, I asked about making an multiplication table in Python. [[a*b for a in x] for b in y] or in NumPy x[..., None]*y[None,...] This is an example of a tensor product between two arrays in order to get a matrix. http://stackoverflow.com/questions/20501073/product-of-arrays-of-different-sizes-in-numpy http://stackoverflow.com/questions/20501073/product-of-array...
- 33a 13y agoThat's a cartesian product, not a tensor product.
- acjohnson55 13y agoI think it might be a bit more correct to call it the outer product.
- mrcactu5 13y agothe outer product is the tensor product of two vectors https://en.wikipedia.org/wiki/Outer_product https://en.wikipedia.org/wiki/Outer_product
- bsaul 13y agoCould that be the start of an introduction to tensor products or even tensors in general ? Start with cartesian product ( or maybe even sql joins ? ) and generalize ? It's probably much more intuitive to us developpers than vector products ( unless you're in 3D graphics or physic engine of course )
- yummyfajitas 13y agoJust one caveat I'd make to his presentation. He says "Naturally, one can extend this definition to n-fold tensor products...", but this isn't actually a definition - it's a theorem. If you wanted to define A x B x C, you could begin by looking at (A x B) x C. This is elements like ((a,b), c) which obey tensor product laws. You could alternately look at A x (B x C), i.e. elements like (a, (b, c)). It's pretty straightforward to prove that there is a natural isomorphism which preserves all the tensor product properties between (A x B) x C and A x (B x C) - it's basically just rearranging the parenthesis. Here the term "natural" also has a technical meaning which agrees completely with it's intuitive one (http://en.wikipedia.org/wiki/Natural_transformation http://en.wikipedia.org/wiki/Natural_transformation). Since (A x B) x C is isomorphic to A x (B x C), it's natural to equate them. You can also define A x B x C in the obvious way, which turns out to be isomorphic to the other two. But it is important to have the theorem proving isomorphism.
- tel 13y agoAnd to amplify this point slightly, the value of this is that we can use a weaker, more general definition and thus capture more behavior with our theory.
- johnlbevan2 13y ago"Tensors are geometric objects that describe linear relations between vectors, scalars, and other tensors": http://en.wikipedia.org/wiki/Tensor http://en.wikipedia.org/wiki/Tensor
- tel 13y agoFor anyone interested in category theory, the description given halfway, that "the tensor product is initial in the category of multilinear maps", is honestly almost enough information to replicate the remainder of this article (the non-computational bits anyway) I'm by no means an expert in CT, but I can now read enough of it to see that. It's kind of like back in high school physics when you knew that everything could be solved by knowing F = ma and doing a bit of algebra.
- gjm11 13y agoUnfortunately, a critically important point in this article is rather misleadingly expressed. The author's first axiom is: "The vectors are v⊗w for v in V, w in W" -- but (as the author clearly realises, and does state elsewhere, but never forcefully enough) the huge majority of elements of V⊗W are not of the form v⊗w. For this reason, I think the author's attempt to present V⊗W as a sort of modified version of V×W is ill-judged. Yes, those are both called products, but they're very different sorts of product; indeed (dropping briefly into the categorical machinery the author introduces at the end) in the category of vector spaces V×W is a coproduct rather than a product. Again, it's clear that the author knows all this. But you wouldn't think so from the early parts of the article. I assume the reason for all this is that the author's trying to relate (scary) tensor products to (nice comfortingly familiar) cartesian products. A laudable aim, but I fear the result will be confusion.
- cscheid 13y agoDouble-plus nitpick: in Vect, VxW can be seen as both a product (which is usually called cartesian product) and a co-product (which is usually called direct sum). So, I guess I would not say "rather than" but "in addition to".
- gjm11 13y agoIt's been a while since I studied category theory so maybe my brain is failing me here, but unless I'm super-confused (1) products are unique up to isomorphism and (2) the product in Vect is the tensor product. The cartesian product is the categorical product in some other categories you can see vector spaces as belonging to -- sets, groups -- but there is only one product in the category of vector spaces and it's the tensor product. What am I missing?
- anonymouz 13y ago(2) is incorrect in that the tensor product is not the category theoretical product in Vect. The product of Vect is given by forming the cartesian product of the vector spaces (with the usual vector space structure on it; and the usual projections into the components are the morphisms you need for the product). The coproduct is the direct sum of vector spaces: In the product construction, take all those vectors where only finitely many coordinates are non-zero (but the morphisms you need for the coproduct are the embeddings of the components into the direct sum). Of course, if you take products or coproducts of finitely many vector spaces, the two coincide. The tensor product is neither a product nor a coproduct in Vect, as the universal property it satisfies is rather different from the product/coproduct ones. (If you consider tensor products of algebras over a commutative base ring R, then the tensor product is the coproduct in the category of R-algebras, but this is probably not what you had in mind.)
- OscarCunningham 13y agoI like this introduction. Often it helps to get used to doing symbol manipulations before trying to fully understand things, and I think tensors are a good example. (By the way, the map "f hat" is at one point described as "multilinear". I think that it's actually linear. After all, it comes out of a tensor product whereas bilinear maps can only really come out of a direct product.)
- yetanotherphd 13y agoWhere? You should point this out so the author can correct it - I couldn't find a point where f-hat was described as multi-linear, which would certainly be a mistake.* It looked to me that f-hat was correctly described as a linear map throughout the article. *Yes, technically a linear map is a multilinear map with rank-1.
- j2kun 13y agoAuthor here: I saw this and should have replied when I did fix it. I'm very appreciative for the feedback from the HN community.
- yetanotherphd 13y agoI see. Great article by the way, I learnt tensor products through physics/exterior algebra, but they tend to teach tensor products and symmetrization/antisymmetrization at the same time. Seeing the plain tensor product was very informative.
- splat 13y agoKip Thorne and Roger Blandford have an excellent introduction to tensors in their lecture notes on the Applications of Classical Mechanics. See sections 1.3 and 1.5: http://www.pma.caltech.edu/Courses/ph136/yr2012/1201.1.K.pdf http://www.pma.caltech.edu/Courses/ph136/yr2012/1201.1.K.pdf They emphasize the geometrical viewpoint in which tensors are just linear functions which map vectors onto vectors via the dot product. (Or more generally a rank-n tensor is a linear function which maps a vector onto a rank n-1 tensor via the dot product.) Throughout the entire course they eschew coordinate systems as much as possible since that tends to obscure the underlying math and physics.
- tedsanders 13y agoI found their introduction to tensors to be the most helpful I had ever read. Realizing that tensors are (or can be considered) functions that map vectors onto vectors was a big conceptual shift for me.
- Patient0 13y agoYes this paper is excellent. Far more comprehensible (to me) than the link at the top of this discussion. I guess it all depends on what you are already familiar with. I was familiar with vectors and functions of vectors, and the concept of a linear function. Therefore, "A rank-n tensor T is, by definition, a real-valued, linear function of n vectors." is a much better starting point for me.
- escape_goat 13y agoI have a grumpy but well founded objection. This may indeed be a very useful guide to tensors, perhaps for people who have worked with them without a strong background understanding. I don't want to detract from that, or from the author's thorough and sincere effort to provide tutelage to the world at large, for free, in a wide variety of mathematical subjects; he seems to have written hundreds of blog posts like this one. However, I don't think that it is in any way about conquering any such thing as 'Tensorphobia', and if it was intended to be "significantly friendly so as not to make anybody run in fear," this effort is going to fail. I do not think that it was; I think there is an (quite innocent) dishonesty to the claim, and I feel like calling this out. The language used contains typical signals of the sort of para-academic elitism parodied in 'Fraiser', starting with the use of the uncommon word 'jest' in the first full sentence and continuing with the adoption of "we" and the scattering of various tofferies throughout remainder of the text. I was quite annoyed by this until I looked at the author's bio page, whereupon I thought, okay, perhaps for him this feels like his natural voice. Making the grade is one of his things, and he has in fact made the grade repeatedly and excessively, and he is young enough that there hasn’t really been much time for him to reflect on this much. He's enthusiastic about teaching, which is a rare gem of a quality amongst the professoriate. So do not be crabby, you old substandard goat, who can barely add two to two without the crutch of an operator. Okay. However, the author's audience is much narrower -- he is making it much narrower -- than he may realize. He is writing for a person like himself, and he is, in a somewhat standard manner, offering the subject by way of little tests of the intellect, couched in suggestions of superiority. This is wonderful fare for a student who's sense of self-worth is founded on the suggestion that they are more clever than the others; it is neither food nor balm for the rest of the anyones who might have cause or need to seek a more general understanding of tensors. What I take exception to is that presenting the subject in this style while claiming to speak to the rest of the anyones will tend to reaffirm 'tensorphobia' for such people, essentially gaslighting them; this is not helpful. It is a distinct harm. I remember realizing, in my first year of university, that a person of ‘general’ (low) intelligence would in fact be entirely capable of understanding the proof of the derivative, if it was taught to them properly. Everyone routinely understands more complicated things. The true barrier would be their indifference; they would have no use for such understanding. But otherwise the failure would not be on the part of the student; it would be on the part of the teacher. I am not suggesting the same of tensors; perhaps I myself lack some quality that is needed to understand them. But I recognize the language that is being used here, and it has an easily comprehensible and distinct meaning that has nothing to do with tensors. I think the author is doing wonderful and useful work, but I think he should reflect long and hard on his rhetorical affectations.
- deleted 13y ago[deleted]
- davyjones 13y agoPrologue: I just searched for the word stress on this essay and couldn't find it. So... * * * Let's see; for me, if I can map an abstract concept to something readily visual, my understanding is faster. Are there some close visual aids to understanding tensors? What physical property(ies?) can be mathematically modeled as a tensor? Imagine a stack of tiles, bottom to top, thin and piled on top of each other. And each tile is connected with its neighbours with springs (not unlike a spring-coiled bed with many layers), like so: ============== ---> Thin tile \ \ \ \ / / / / ---> Springs \ \ \ \ ============== \ \ \ \ / / / / \ \ \ \ ============== \ \ \ \ / / / / \ \ \ \ ============== Now, we can pull the topmost tile along the stacked direction causing the springs to expand. If we do this and only this, it is pure tensile stress (I am referring to stress in a bit loose way here). We can also sit on that stack and that leads to compressive stress (just a tensile stress with a minus sign). [As a sidenote, bricks can take great compressive forces but can't withstand tensile forces of similar levels. But something like steel has almost symmetric response between tensile and compressive loads]. OK...what else can we do? Can I pull the topmost tile to the right (or left) while holding the bottommost tile still? Sure, and now the stack looks like a rhombus. This is shear stress...in the right-left direction. I could've also pulled the topmost tile towards (or away from) me. That is also shear stress in the front-back direction. So, for this setup, we can identify three stress components: 1 tensile and 2 shear. Now, the first tricky bit: imagine a "stack" that is bottom-top, left-right and front-back. There are springs running in all three directions. We have 3 tensile and 6 shear components. Second tricky bit: shrink that new whole "stack" to a point. We have the stress tensor(!). s = [s_11 s_12 s_13, s_21 s_22 s_23, s_31 s_32 s_33] Tensors are used all the time in mechanics. Very very useful stuff.
- deleted 13y ago[deleted]
- shiven 13y agoThis is a much clearer explanation than the original blog post (it used terms without explaining them to non-mathematicians i.e. me). Thank you!
- 13y ago
- swordswinger12 13y agoThere's some recent work in cryptography on multilinear maps! Fascinating stuff.
- midas007 13y agoIIRC tensors are one of the fundamental prereqs for understanding General Relativity. (Bucketlist topic.)