7 ms·

That was explanation from a perspective of someone acquainted with modern physics. As such, it will make sense to physicist, but no sense to most everyone else,

by xyzzyz 5y ago

That was explanation from a perspective of someone acquainted with modern physics. As such, it will make sense to physicist, but no sense to most everyone else, including mathematicians who don’t know modern physics. For example, in the beginning, author describes tensors as things behaving according to tensor transformation formula. This is already very much a physicist kind of thinking: it assumes that there is some object out there, and we’re trying to understand what it is in terms of how it behaves. It also uses the summation notation which is rather foreign to non-physicist mathematicians. Then, when it finally reaches the point where it is all related to tensors in TensorFlow sense, we find that there is no reference made to the transformation formula, purportedly so crucial to understanding tensors. How comes? The solution here is quite simple: what author (and physicists) call tensors is not what TensorFlow (and mathematicians) call tensors. Instead, author describes what mathematicians call “a tensor bundle”, which is a correspondence that assigns each point of space a unique tensor. That’s where the transformation rule comes from: if we describe this mapping in terms of some coordinate system (as physicist universally do), the transformation rule tells you how to this description changes in terms of change of the coordinates. This setup, of course, has little to do with TensorFlow, because there is no space that its tensors are attached to, they are just standalone entities. So what are the mathematician’s (and TensorFlow) tensors? They’re actually basically what the author says, after very confusing and irrelevant introduction talking about change of coordinates of underlying space — irrelevant, because TensorFlow tensors are not attached as a bundle to some space (manifold) as they are on physics, so no change of space coordinates ever happens. Roughly, tensors are a sort of universal objects representing multi linear maps: bilinear maps V x W -> R correspond canonically one-to-one to regular linear maps V (x) W -> R, where V (x) W is a vector space called tensor product of V and W, and tensors are simply vectors in this tensor product space. Basically, the idea is to replace weird multi linear objects with normal linear objects (vectors), that we know how to deal with, using matrix multiplication and stuff. That’s all there is to it.

mbbutler 5y ago
Why are you complaining that the author didn't talk about tensors as they are used in tensorflow? Tensorflow is never even mentioned in the piece. The author is perfectly clear in the first sentence that the piece's focus is about the usefulness of tensors in a physics context.
- xyzzyz 5y ago
  Huh, you’re right, not sure why I thought it’s TensorFlow related.
ABeeSea 5y ago
> Instead, author describes what mathematicians call “a tensor bundle”, which is a correspondence that assigns each point of space a unique tensor. Technically, that’s a tensor field which is a section of the tensor bundle. Similarly, a vector field is a section of the tangent bundle (the collection of all the tangent spaces of the points on the manifold). A vector field is just a choice of a tangent vector for each point from that point’s tangent space.
hansen 5y ago
To be a bit pedantic: the identification of tensors with multilinear forms requires finite dimensions (or reflexive topological spaces).
hinkley 5y ago
> author describes tensors as things behaving according to tensor transformation formula In grade school it drove me nuts when the homework required us to describe a word without using the word (or it’s Latinate siblings). And yet as an adult there are few enough weeks that go by where some grownup doesn’t try to pull that same trick. If you think developers are guilty of circular logic, check out some of the math pages on Wikipedia. You can get lost in moments.
- jhokanson 5y ago
  Speaking of math pages on Wikipedia ... and math text more generally Is it just me or are we horrible at teaching advanced math? Where are the examples (with actual numbers)? Where is the motivation? Where are the pictures?
  - chobytes 5y ago
    This is definitely a problem! Having a large set of interests and problems to draw examples and intuition from are how I deal with it. I suspect this is why so many mathematicians are also into physics.
    - exdsq 5y ago
      100%! For those of us who need to learn from practical examples through to generalized intuition maths can be really really hard to learn depending on the source. Wish I was one of those people who finds it easier to learn from abstract first through to implementations second.
  - hinkley 5y ago
    Randall Monroe has a comic about how most people need enough math to be able to handle a birthday dinner where the guests split the bill for the birthday boy/girl evenly and pay for their meals and tip separately. That’s a pretty good bar and I wonder if we could just cut to that chase earlier. But I also believe that people need enough math to see when they’re being cheated, and I feel like you could just tell middle schoolers that and they would pay attention. Maybe even primary school. You told Billy he could have three apples, and now there are two left. Did Billy take more apples than he should have? It’s always how do you share your cookies fairly with your friends and if they’re my cookies why do I have to share them at all? Screw “fairly” I’m keeping the extras at least. That sort of sharing is a socially advanced concept they don’t entirely get just yet.
tagrun 5y ago
It has nothing to do with tensor fields, uniform/constant tensors still obey the proper coordinate transformations, that's the defining property of any tensor. (With non-uniform tensor fields, covariant derivatives also pick up a correction, but that's a separate thing.) TensorFlow "tensor"(and most other use of "tensor" in programmer jargon) is not a tensor at all, it's just a multidimensional array.
- catgary 5y ago
  What do you think the tensor product of finite dimensional vector spaces looks like?
- contravariant 5y ago
  Mathematicians would disagree with you there. There are no coordinates to transform in an ordinary tensor space and therefore no way for a tensor to be affected by such a transformation. Matrices (or linear transformations in general) are important examples of tensors. There's a nice adjunction between tensor spaces A(x)B and the space of linear transformations B=>C given by: Hom(A(x)B, C) = Hom(A, B=>C) In the case of Tensorflow I think they do actually still talk about linear transformations of some kind so it's perfectly fine to call them tensors.
  - cygx 5y ago
    There are no coordinates to transform in an ordinary tensor space and therefore no way for a tensor to be affected by such a transformation. Sure there are: Any basis of the underlying vector space(s) induces a basis of the tensor space. Components respective to some basis are coordinates. You can then investigate what happens to the induced basis (or rather, the respective components) under a basis transformation of the underlying vector space(s), which is where the "physicist's" definition of tensors originates.
    - contravariant 5y ago
      The components of a vector aren't the same as the coordinates physicists talk about when they're dealing with tensors. The components would be something like the value of the magnetic potential, or the local wind speed. The coordinates would be the location where that particular vector is 'anchored'. A change of coordinates does indeed induce a change of basis, but a change of basis isn't really a change of coordinates. And strictly speaking some vector spaces don't really have an obvious basis (without invoking choice), so having a basis be a prerequisite for the definition is not ideal. The whole requirement that a tensor is 'something that transforms like [...] under a coordinate transformation' is just how physicists have chosen to phrase that a vector bundle is only well defined if it's definition isn't dependent on some arbitrary choice of coordinates. In my opinion this requirement is more easily apparent in the mathematical definition where there is no choice of coordinates in the first place, rather than the physicists way of working with some choice of coordinates and checking how things transform.
mr_mitm 5y ago
Thanks for this summary. Even as a physicist I found it highly confusing when I got told in physics classes that a tensor is "just a thing (or object) that behaves like so under coordinate transformation". Like, what do you mean by "thing"? I have no intuition to this yet, I need it concise definitions! Fortunately I took a differential geometry class at the same time, which was really helpful.
ummonk 5y ago
The post was also a poor explanation for someone doing modern physics. [edit: not true actually I should have read the rest of the post - it’s a good post] Wald's approach in General Relativity is much better - he treats Tensors as a multilinear map from vectors and dual vectors to scalars. He then derives the underlying coordinate transformaton rules, for the vector spaces used in differential geometry. But
- mfn 5y ago
  That’s the approach I used as well in the second half of the article - I just mentioned the transformation law in the beginning since that’s what most physics students encounter first. Most of the article tries to provide some intuition behind why multilinear maps, which sound like a fairly abstract concept, might be relevant in physics. The key link being the importance of coordinate invariance. I didn’t go into deriving the coordinate transforms from the multilinear map definition as I didn’t feel that it’d provide much better intuition, but I did mention the equivalence near the end.
  - ummonk 5y ago
    Yeah sorry you’re right - I should have read the rest of your post, which is excellent and describes precisely why the coordinates/transformations focused definition is bad for one’s intuition.
kaashif 5y ago
> author describes tensors as things behaving according to tensor transformation formula Yeah, the idea that there are pre-existing things that we're trying to describe is somewhat weird to me when we're trying to come up with a definition of a tensor. The whole point of mathematics is that you come up with the definitions and theorems fall out. In particular, this comment is funny and speaks to some difference in how I and the author view what we're doing when defining a tensor: > But why that specific transformation law - why must tensors transform in that way in order to preserve whatever object the tensor represents? Because we defined it like that! When you make the definition "a tensor is a thing that follows X laws", you don't get to ask why, you just defined it! Just a funny bit of phrasing, I get what is meant :)
- edflsafoiewq 5y ago
  > The whole point of mathematics is that you come up with the definitions and theorems fall out. That's just how it's presented in textbooks. It's obviously not math is actually done.
  - ravi-delia 5y ago
    It's presented that way in textbooks because it's way easier to learn it that way (or it's Stockholm syndrome, which I won't deny is possible). The motivated way would require way, way, way more background knowledge.
bigger_cheese 5y ago
Engineering usage seems to match the physics usage. In classic engineering fashion however we were always taught just to 'plug them in' without learning all the minutia that go with them. For example the stress and strain calculations which are used for calculating Deformation (Say if you were rolling a sheet of steel in a mill) makes use of tensors and also something called an "Invariant" I assume this also comes from Physics/Mathematics world.
l33t2328 5y ago
I think it’s a little funny you said that >[the explanation in the OP] will make sense to physicist, but no sense to most everyone else, including mathematicians And then went on to describe tensors in a way that is unfriendly to non mathematicians by saying > tensors are a sort of universal objects representing multi linear maps: bilinear maps V x W -> R correspond canonically one-to-one to regular linear maps V (x) W -> R, where V (x) W is a vector space called tensor product of V and W, and tensors are simply vectors in this tensor product space.

by xyzzyz 5y ago

mbbutler 5y ago

Why are you complaining that the author didn't talk about tensors as they are used in tensorflow? Tensorflow is never even mentioned in the piece. The author is perfectly clear in the first sentence that the piece's focus is about the usefulness of tensors in a physics context.

xyzzyz 5y ago
Huh, you’re right, not sure why I thought it’s TensorFlow related.

ABeeSea 5y ago

> Instead, author describes what mathematicians call “a tensor bundle”, which is a correspondence that assigns each point of space a unique tensor. Technically, that’s a tensor field which is a section of the tensor bundle. Similarly, a vector field is a section of the tangent bundle (the collection of all the tangent spaces of the points on the manifold). A vector field is just a choice of a tangent vector for each point from that point’s tangent space.

hansen 5y ago

To be a bit pedantic: the identification of tensors with multilinear forms requires finite dimensions (or reflexive topological spaces).

hinkley 5y ago

> author describes tensors as things behaving according to tensor transformation formula In grade school it drove me nuts when the homework required us to describe a word without using the word (or it’s Latinate siblings). And yet as an adult there are few enough weeks that go by where some grownup doesn’t try to pull that same trick. If you think developers are guilty of circular logic, check out some of the math pages on Wikipedia. You can get lost in moments.

jhokanson 5y ago
Speaking of math pages on Wikipedia ... and math text more generally Is it just me or are we horrible at teaching advanced math? Where are the examples (with actual numbers)? Where is the motivation? Where are the pictures?
- chobytes 5y ago
  This is definitely a problem! Having a large set of interests and problems to draw examples and intuition from are how I deal with it. I suspect this is why so many mathematicians are also into physics.
  - exdsq 5y ago
    100%! For those of us who need to learn from practical examples through to generalized intuition maths can be really really hard to learn depending on the source. Wish I was one of those people who finds it easier to learn from abstract first through to implementations second.
- hinkley 5y ago
  Randall Monroe has a comic about how most people need enough math to be able to handle a birthday dinner where the guests split the bill for the birthday boy/girl evenly and pay for their meals and tip separately. That’s a pretty good bar and I wonder if we could just cut to that chase earlier. But I also believe that people need enough math to see when they’re being cheated, and I feel like you could just tell middle schoolers that and they would pay attention. Maybe even primary school. You told Billy he could have three apples, and now there are two left. Did Billy take more apples than he should have? It’s always how do you share your cookies fairly with your friends and if they’re my cookies why do I have to share them at all? Screw “fairly” I’m keeping the extras at least. That sort of sharing is a socially advanced concept they don’t entirely get just yet.

tagrun 5y ago

It has nothing to do with tensor fields, uniform/constant tensors still obey the proper coordinate transformations, that's the defining property of any tensor. (With non-uniform tensor fields, covariant derivatives also pick up a correction, but that's a separate thing.) TensorFlow "tensor"(and most other use of "tensor" in programmer jargon) is not a tensor at all, it's just a multidimensional array.

catgary 5y ago
What do you think the tensor product of finite dimensional vector spaces looks like?
contravariant 5y ago
Mathematicians would disagree with you there. There are no coordinates to transform in an ordinary tensor space and therefore no way for a tensor to be affected by such a transformation. Matrices (or linear transformations in general) are important examples of tensors. There's a nice adjunction between tensor spaces A(x)B and the space of linear transformations B=>C given by: Hom(A(x)B, C) = Hom(A, B=>C) In the case of Tensorflow I think they do actually still talk about linear transformations of some kind so it's perfectly fine to call them tensors.
- cygx 5y ago
  There are no coordinates to transform in an ordinary tensor space and therefore no way for a tensor to be affected by such a transformation. Sure there are: Any basis of the underlying vector space(s) induces a basis of the tensor space. Components respective to some basis are coordinates. You can then investigate what happens to the induced basis (or rather, the respective components) under a basis transformation of the underlying vector space(s), which is where the "physicist's" definition of tensors originates.
  - contravariant 5y ago
    The components of a vector aren't the same as the coordinates physicists talk about when they're dealing with tensors. The components would be something like the value of the magnetic potential, or the local wind speed. The coordinates would be the location where that particular vector is 'anchored'. A change of coordinates does indeed induce a change of basis, but a change of basis isn't really a change of coordinates. And strictly speaking some vector spaces don't really have an obvious basis (without invoking choice), so having a basis be a prerequisite for the definition is not ideal. The whole requirement that a tensor is 'something that transforms like [...] under a coordinate transformation' is just how physicists have chosen to phrase that a vector bundle is only well defined if it's definition isn't dependent on some arbitrary choice of coordinates. In my opinion this requirement is more easily apparent in the mathematical definition where there is no choice of coordinates in the first place, rather than the physicists way of working with some choice of coordinates and checking how things transform.

mr_mitm 5y ago

Thanks for this summary. Even as a physicist I found it highly confusing when I got told in physics classes that a tensor is "just a thing (or object) that behaves like so under coordinate transformation". Like, what do you mean by "thing"? I have no intuition to this yet, I need it concise definitions! Fortunately I took a differential geometry class at the same time, which was really helpful.

ummonk 5y ago

The post was also a poor explanation for someone doing modern physics. [edit: not true actually I should have read the rest of the post - it’s a good post] Wald's approach in General Relativity is much better - he treats Tensors as a multilinear map from vectors and dual vectors to scalars. He then derives the underlying coordinate transformaton rules, for the vector spaces used in differential geometry. But

mfn 5y ago
That’s the approach I used as well in the second half of the article - I just mentioned the transformation law in the beginning since that’s what most physics students encounter first. Most of the article tries to provide some intuition behind why multilinear maps, which sound like a fairly abstract concept, might be relevant in physics. The key link being the importance of coordinate invariance. I didn’t go into deriving the coordinate transforms from the multilinear map definition as I didn’t feel that it’d provide much better intuition, but I did mention the equivalence near the end.
- ummonk 5y ago
  Yeah sorry you’re right - I should have read the rest of your post, which is excellent and describes precisely why the coordinates/transformations focused definition is bad for one’s intuition.

kaashif 5y ago

> author describes tensors as things behaving according to tensor transformation formula Yeah, the idea that there are pre-existing things that we're trying to describe is somewhat weird to me when we're trying to come up with a definition of a tensor. The whole point of mathematics is that you come up with the definitions and theorems fall out. In particular, this comment is funny and speaks to some difference in how I and the author view what we're doing when defining a tensor: > But why that specific transformation law - why must tensors transform in that way in order to preserve whatever object the tensor represents? Because we defined it like that! When you make the definition "a tensor is a thing that follows X laws", you don't get to ask why, you just defined it! Just a funny bit of phrasing, I get what is meant :)

edflsafoiewq 5y ago
> The whole point of mathematics is that you come up with the definitions and theorems fall out. That's just how it's presented in textbooks. It's obviously not math is actually done.
- ravi-delia 5y ago
  It's presented that way in textbooks because it's way easier to learn it that way (or it's Stockholm syndrome, which I won't deny is possible). The motivated way would require way, way, way more background knowledge.

bigger_cheese 5y ago

Engineering usage seems to match the physics usage. In classic engineering fashion however we were always taught just to 'plug them in' without learning all the minutia that go with them. For example the stress and strain calculations which are used for calculating Deformation (Say if you were rolling a sheet of steel in a mill) makes use of tensors and also something called an "Invariant" I assume this also comes from Physics/Mathematics world.

l33t2328 5y ago

I think it’s a little funny you said that >[the explanation in the OP] will make sense to physicist, but no sense to most everyone else, including mathematicians And then went on to describe tensors in a way that is unfriendly to non mathematicians by saying > tensors are a sort of universal objects representing multi linear maps: bilinear maps V x W -> R correspond canonically one-to-one to regular linear maps V (x) W -> R, where V (x) W is a vector space called tensor product of V and W, and tensors are simply vectors in this tensor product space.