12 ms·
Why tensors? A beginner's perspective
- saberience 5y agoThis doesn't seem like it's for beginners.
- VeninVidiaVicii 5y ago> Most commonly, a tensor is defined as being anything that transforms like a tensor. Definitely not beginner level.
- brummm 5y agoHmm, this is stuff physicists learn in their first year undergrad classes for mathematical foundations. Seems to me it's the very definition of beginner.
- cyber_kinetist 5y agoI don't know what undergraduate program you have gone through, but this is definitely second-year or third-year course material for most physics degrees in universities. Maybe if you've already taken lots of AP classes in high school then you might be able to skip some stuff, but we're talking about the standard curriculum here. Normally, you first study the distinction between vectors (which can be expanded to tensors) and scalars in second-year Analytical Mechanics class. You also get a taste of tensors toward the later material in Electromagnetism (which is also probably second-year). And you finally arrive at a rigorous definition of tensors when you take Mathematical Physics (second-year or third-year depending on your skills).
- ericphanson 5y agoI was happy to see that this article is actually talking about tensors, not just multidimensional arrays (which for some reasons are often called tensors by machine learning folks).
- The_rationalist 5y agoWhat is the difference?
- icapybara 5y agoTensors have additional properties that arrays don't necessarily have. For example, the coordinate system transform rule that the author describes in the beginning of the post. One of my old physics professors taught us to think of tensors as "arrays with units." If it's a vector/matrix/higher dimensional array but has physical units, it's probably a tensor. The fact that it has units means it represents something physical which must obey additional constraints (like the coordinate system transformation rule).
- omarhaneef 5y agoSince I also thought Tensors were just higher dimension arrays, isn't this really what ML folks think Tensors are, since they (we?) do attach units to the Tensors most of the time?
- ogogmad 5y agoThe physicist's approach is a bit non-conceptual. From a mathematical point of view, a tensor is essentially an arbitrary multi-linear map. Think of the dot product, the determinant of a matrix (which is linear on each column but is not linear on a matrix), the exterior product in exterior algebra (or geometric algebra), a linear map itself (which is obviously a special case of a multilinear map), etc. The coordinate change stuff that physicists talk about stems from observing that a matrix can be used to represent some tensors, but the rule for changing basis changes along with the kind of tensor. So if M is a matrix which represents a linear map and P is a matrix whose columns are basis vectors, then PMP^{-1} is the same linear map as M but in basis P; if on the other hand the matrix M represents a bilinear form as opposed to a linear map, then the basis change formula is actually PMP^T, where we use the matrix transpose. Sylvester's Law Of Inertia is then a non-trivial observation about matrix representations of bilinear forms. Physicists conflate a tensor with its representation in some coordinate system. Then they show how changing the coordinate system changes the coordinates. This point of view does provide some concrete intuition, though, so it's not all bad. By a coordinate system, I mean a linear basis. Hope that helps.
- beaconstudios 5y agoOK that helps me to understand why tensorflow is called what it is - if a tensor turns a set of vectors into a scalar that's exactly what an artificial neuron does with weights and inputs, and they are linked up to form a data flow graph.
- bmitc 5y agoAnyone interested in a visual exploration should checkout Geometrical Vectors by Gabriel Weinreich. https://www.maa.org/press/maa-reviews/geometrical-vectors https://www.maa.org/press/maa-reviews/geometrical-vectors
- billfruit 5y agoIs there any book that treats whole off geometry using vectors?
- bmitc 5y agoI’m not sure I understand the question enough to answer. Do you mean something like differential geometry? There, the theory is built upon vectors and covectors (i.e., differential forms) that are associated with tangent spaces and cotangent spaces, respectively. But that is modern differential geometry and not classical geometry.
- billfruit 5y agoI was asking for a classical geometry book which is using a treatment using vectors. Usually classical geometry is treated without resorting to vectors.
- bmitc 5y agoI'm still not really sure what you mean. The most "modern" treatment of classical geometry that I know of is Geometry by Brannan, Esplen, Gray. It might be worth a look.
- billfruit 5y agoThat is still not using vectors as much. My question is if classical geometric problems can be solved using vectors, using vector concepts for example using cross products for area expressions?
- Koshkin 5y agoHere is a really good resource for a beginner: https://grinfeld.org/books/An-Introduction-To-Tensor-Calculus/ https://grinfeld.org/books/An-Introduction-To-Tensor-Calculu...
- ok123456 5y agoIs that you Pavel?
- xyzzyz 5y agoThat was explanation from a perspective of someone acquainted with modern physics. As such, it will make sense to physicist, but no sense to most everyone else, including mathematicians who don’t know modern physics. For example, in the beginning, author describes tensors as things behaving according to tensor transformation formula. This is already very much a physicist kind of thinking: it assumes that there is some object out there, and we’re trying to understand what it is in terms of how it behaves. It also uses the summation notation which is rather foreign to non-physicist mathematicians. Then, when it finally reaches the point where it is all related to tensors in TensorFlow sense, we find that there is no reference made to the transformation formula, purportedly so crucial to understanding tensors. How comes? The solution here is quite simple: what author (and physicists) call tensors is not what TensorFlow (and mathematicians) call tensors. Instead, author describes what mathematicians call “a tensor bundle”, which is a correspondence that assigns each point of space a unique tensor. That’s where the transformation rule comes from: if we describe this mapping in terms of some coordinate system (as physicist universally do), the transformation rule tells you how to this description changes in terms of change of the coordinates. This setup, of course, has little to do with TensorFlow, because there is no space that its tensors are attached to, they are just standalone entities. So what are the mathematician’s (and TensorFlow) tensors? They’re actually basically what the author says, after very confusing and irrelevant introduction talking about change of coordinates of underlying space — irrelevant, because TensorFlow tensors are not attached as a bundle to some space (manifold) as they are on physics, so no change of space coordinates ever happens. Roughly, tensors are a sort of universal objects representing multi linear maps: bilinear maps V x W -> R correspond canonically one-to-one to regular linear maps V (x) W -> R, where V (x) W is a vector space called tensor product of V and W, and tensors are simply vectors in this tensor product space. Basically, the idea is to replace weird multi linear objects with normal linear objects (vectors), that we know how to deal with, using matrix multiplication and stuff. That’s all there is to it.
- mbbutler 5y agoWhy are you complaining that the author didn't talk about tensors as they are used in tensorflow? Tensorflow is never even mentioned in the piece. The author is perfectly clear in the first sentence that the piece's focus is about the usefulness of tensors in a physics context.
- 725686 5y agoA wonderful little video to understand what tensors are, by Daniel Fleish: https://www.youtube.com/watch?v=f5liqUk0ZTw https://www.youtube.com/watch?v=f5liqUk0ZTw Very simple and basic. Edit: incorrectly wrote vectors instead of tensors.
- mettamage 5y agoWow, that's such a good video. Thanks! Haha, mind blown really. And other than graph theory, I never took a college level math course (I artfully skipped almost all math during my CS degree), I'm doing pre-calculus at the moment, because I want to get better at it.
- chobytes 5y agoMy version is just: Tensors allow us to write data and operations on data in a way which does not depend on how we chose to represent them. For example, if I have a vector x in V and a map T from V to W, then I would like the truth of T(x)=y to be independent of how I represent T and x.
- zardo 5y agoI like the concrete example from when I first used tensors in school. Stress in a block of concrete. You can choose any basis you like to represent the stresses and transform between them. Whether or not the concrete block breaks under that stress obviously does not depend on your choice of basis or units, so your transformation rules had better reflect that reality.
- Beldin 5y agoThe way I think of it: you have 0-dimensional arrays of numbers (plain numbers or scalars). You have 1-dimensional arrays of numbers (a list of N numbers or an N-vector). You have 2-dimensional arrays of numbers (an NxM matrix). We can extend this concept to 3- and 4-dimensional arrays and even further. The kicker? All of them are tensors. Tensor is just a generalisation of the concept. I am no licensed mathematician, so this could be off. However, every time I dive into this topic, I have to wade through way too complex mathnobabble to arrive at that notion. So let's keep it simple: tensors are a mathematician's template for arrays of any dimension.
- mkehrt 5y agoA (d_0 * d_1 * ... * d_{k-1} * d_k) tensor is just a linear map from a (d_0 * d_1 * ... * d_{m-1} * d_{m+1} * ... * d_{k-1} * d_k) tensor to a (d_0 * d_1 * ... * d_{n-1} * d_{n+1} * ... * d_{k-1} * d_k) tensor, where a () tensor is a scalar, right? (I kid, but I think this is true, right?)
- suydyswjuddyy 5y ago
- steve76 5y ago