4 ms·

Tensors are most definitely not just multidimensional arrays. (this is a pet peeve of mine) Tensors have geometric meaning just like vectors (just like vectors

Tensors are most definitely not just multidimensional arrays. (this is a pet peeve of mine) Tensors have geometric meaning just like vectors (just like vectors are not just single-axis arrays). It would be too long to delve into the geometry (especially if it is abstract "geometry" like the geometrical space spanned by the songs in the Pandora database or the space spanned by the faces in a face-recognition software). Instead I will try to give an example: You all know what rotating a vector means. All the components change in certain way, but the geometrical meaning of this object is preserved in some sense. This is not true for some arbitrary list of numbers. Well, a tensor is just a more complicated geometrical structure that still has very specific rules governing what its components can be and how they can change. And a counterexample: the Christoffel Symbol is a multi-dimensional array that is not a tensor. P.S. It is true that it is a very common abuse of nomenclature to call all multidimensional arrays "tensors".

chestervonwinch 12y ago
It's true though that vectors are basically 1D lists if we're speaking about elements from the vector space R^n. Is it true that tensors are multidimensional arrays if they come from the space R^{m1 x m2 x ... x mn} ? I'm also confused: do you think about a tensor as a vector from vector space? as a object that maps between vector spaces? or something completely different? I realize there's a LOT of detail here - just trying to flesh out my 500,000,000 mile view a little more.
- tel 12y ago
  It's not true, though. A list of numbers in R is not a vector until you imbue the position of each of those numbers with a geometric basis, or, alternatively, relate that list of numbers to the set of all lists of such numbers along with some algebraic operations which turn it into a vector space. This is similar with tensors. It's even harder to talk about them, though, without the geometric/algebraic bits.
- im3w1l 12y ago
  They use tensors here to mean simply multidimensional array. And as this discussion shows, that choice of words is unfortunate and confusing.
  - rspeer 12y ago
    What I'm seeing is that physicists and geometers disagree with programmers about what "tensor" means -- not in that their views are inconsistent, but in that the programmers' view is too limited for the things that the physicists and geometers want to do with tensors. They also disagree about what "vector" and "matrix" mean for the same reason, but they can't really put that horse back in the barn. But "tensors" remain unfamiliar enough that this disagreement can come up whenever anyone says "tensor". Here's a possible truce: - Programmers should default to talking about "arrays" - Geometers and physicists should default to talking about "vectors", "matrices", and "tensors" - We all generally recognize that some (vectors|matrices|tensors) are arrays, and some arrays are (vectors|matrices|tensors), and it's not the worst thing ever if someone uses one to mean the other - We all generally recognize that you can sometimes do damn cool things by using an array as a tensor or a tensor as an array, so there isn't that much benefit in shouting "NO THEY'RE TOTALLY DIFFERENT".
    - tel 12y ago
      Eh, I think as you stack more and more "structure" atop something it becomes less and less appropriate to confuse it with its representation. There is afterall already a completely appropriate, cross-disciplinary term for what "programmers" here call "tensors": multidimensional arrays, or even nd-arrays. No mathematician will bat an eye if you say that a tensor can be represented by an nd-array---but if you equate them, then you're clearly missing something.
      - dvanduzer 12y ago
        So... Do "programmers" think of "matrix" and "tensor" as "synonymous"? edit: that was actually a serious question, because i really don't understand this idea that programmers have a common notion of what a matrix is.
- musgravepeter 12y ago
  A 1D list of numbers is not necessarily a "vector" in the Differential Geometry sense. The definition of a vector or tensor is based on how it transforms as the co-ordinate system transforms. See e.g. http://en.wikipedia.org/wiki/Covariance_and_contravariance_of_vectors http://en.wikipedia.org/wiki/Covariance_and_contravariance_o... One weird consequence of this is that if A and B are vectors, the cross product (AxB) is NOT a vector in the tensorial sense. [Aside: Cross products are really weird since they are defined in the familiar way 3 and 7 dimensions only! (see http://math.stackexchange.com/questions/185991/is-the-vector-cross-product-only-defined-for-3d) http://math.stackexchange.com/questions/185991/is-the-vector...]
- evanpw 12y ago
  The word "vector" just means an element of a vector space: a collection of "things" that you can add together and multiply by numbers. The vector space R^n is defined as the collection of all lists of n real numbers, so you are right in that sense. But usually when people say that a vector "is" a list of numbers, they mean that if you choose a coordinate system, then the coordinates of a vector uniquely identify it. This is exactly analogous to locations on the earth being uniquely identified by their latitude and longitude: we first have to make a (somewhat arbitrary) choice of coordinates. You wouldn't say that a point on the earth "is" a pair of numbers, though. The case of R^n is confusing, because if you choose the usual coordinate system, then the coordinates of a vector are the same list of numbers as the vector itself. I like to think of tensors as functions that map some number of vectors to some other number of vectors. But you can add together two tensors (with the same number of inputs and outputs) or multiply a tensor by a number, so you can also think of them as vectors in some other vector space.
  - platz 11y ago
    Almost seems like tensors are natural transformations in the category theoretic sense.
evanpw 12y ago
I'm not sure if I'm disagreeing with you, but I really dislike the "collection of numbers that transforms in this way" definition of vectors, tensors, etc. I get why it's useful in calculations (that's why physicists like it), but it seems really inelegant to me. To me, a tensor is a function that maps a collection of m vectors to a collection of n vectors, such that every output is linear in each input. It's true that if you choose a basis, then you can write down an array of numbers which identifies the function, and that changing the basis causes those numbers to change in a certain way, but that's not what a tensor "is". Of course, the beauty of mathematics is that there are many different ways of looking at the same object, so physicists can deal with their arrays of numbers, I can play with my multilinear functions, and everyone gets the same answer when we ask the same question.
- cygx 12y ago
  I really dislike the "collection of numbers that transforms in this way" definition of vectors, tensors, etc. It is not totally without merit if you consider the tensor bundle as associated to the principal bundle of linear frames. However, that's hardly suitable as an introduction to the topic.
  - tel 12y ago
    It's probably the Wikipedia introduction, though...
Enzolangellotti 12y ago
So, a tensor is kind of a Rubik Cube shaped matrix?
- ajkjk 12y ago
  A rank-3 tensor is. It's any number of dimensions you need.

chestervonwinch 12y ago

It's true though that vectors are basically 1D lists if we're speaking about elements from the vector space R^n. Is it true that tensors are multidimensional arrays if they come from the space R^{m1 x m2 x ... x mn} ? I'm also confused: do you think about a tensor as a vector from vector space? as a object that maps between vector spaces? or something completely different? I realize there's a LOT of detail here - just trying to flesh out my 500,000,000 mile view a little more.

tel 12y ago
It's not true, though. A list of numbers in R is not a vector until you imbue the position of each of those numbers with a geometric basis, or, alternatively, relate that list of numbers to the set of all lists of such numbers along with some algebraic operations which turn it into a vector space. This is similar with tensors. It's even harder to talk about them, though, without the geometric/algebraic bits.
im3w1l 12y ago
They use tensors here to mean simply multidimensional array. And as this discussion shows, that choice of words is unfortunate and confusing.
- rspeer 12y ago
  What I'm seeing is that physicists and geometers disagree with programmers about what "tensor" means -- not in that their views are inconsistent, but in that the programmers' view is too limited for the things that the physicists and geometers want to do with tensors. They also disagree about what "vector" and "matrix" mean for the same reason, but they can't really put that horse back in the barn. But "tensors" remain unfamiliar enough that this disagreement can come up whenever anyone says "tensor". Here's a possible truce: - Programmers should default to talking about "arrays" - Geometers and physicists should default to talking about "vectors", "matrices", and "tensors" - We all generally recognize that some (vectors|matrices|tensors) are arrays, and some arrays are (vectors|matrices|tensors), and it's not the worst thing ever if someone uses one to mean the other - We all generally recognize that you can sometimes do damn cool things by using an array as a tensor or a tensor as an array, so there isn't that much benefit in shouting "NO THEY'RE TOTALLY DIFFERENT".
  - tel 12y ago
    Eh, I think as you stack more and more "structure" atop something it becomes less and less appropriate to confuse it with its representation. There is afterall already a completely appropriate, cross-disciplinary term for what "programmers" here call "tensors": multidimensional arrays, or even nd-arrays. No mathematician will bat an eye if you say that a tensor can be represented by an nd-array---but if you equate them, then you're clearly missing something.
    - dvanduzer 12y ago
      So... Do "programmers" think of "matrix" and "tensor" as "synonymous"? edit: that was actually a serious question, because i really don't understand this idea that programmers have a common notion of what a matrix is.
musgravepeter 12y ago
A 1D list of numbers is not necessarily a "vector" in the Differential Geometry sense. The definition of a vector or tensor is based on how it transforms as the co-ordinate system transforms. See e.g. http://en.wikipedia.org/wiki/Covariance_and_contravariance_of_vectors http://en.wikipedia.org/wiki/Covariance_and_contravariance_o... One weird consequence of this is that if A and B are vectors, the cross product (AxB) is NOT a vector in the tensorial sense. [Aside: Cross products are really weird since they are defined in the familiar way 3 and 7 dimensions only! (see http://math.stackexchange.com/questions/185991/is-the-vector-cross-product-only-defined-for-3d) http://math.stackexchange.com/questions/185991/is-the-vector...]
evanpw 12y ago
The word "vector" just means an element of a vector space: a collection of "things" that you can add together and multiply by numbers. The vector space R^n is defined as the collection of all lists of n real numbers, so you are right in that sense. But usually when people say that a vector "is" a list of numbers, they mean that if you choose a coordinate system, then the coordinates of a vector uniquely identify it. This is exactly analogous to locations on the earth being uniquely identified by their latitude and longitude: we first have to make a (somewhat arbitrary) choice of coordinates. You wouldn't say that a point on the earth "is" a pair of numbers, though. The case of R^n is confusing, because if you choose the usual coordinate system, then the coordinates of a vector are the same list of numbers as the vector itself. I like to think of tensors as functions that map some number of vectors to some other number of vectors. But you can add together two tensors (with the same number of inputs and outputs) or multiply a tensor by a number, so you can also think of them as vectors in some other vector space.
- platz 11y ago
  Almost seems like tensors are natural transformations in the category theoretic sense.

evanpw 12y ago

I'm not sure if I'm disagreeing with you, but I really dislike the "collection of numbers that transforms in this way" definition of vectors, tensors, etc. I get why it's useful in calculations (that's why physicists like it), but it seems really inelegant to me. To me, a tensor is a function that maps a collection of m vectors to a collection of n vectors, such that every output is linear in each input. It's true that if you choose a basis, then you can write down an array of numbers which identifies the function, and that changing the basis causes those numbers to change in a certain way, but that's not what a tensor "is". Of course, the beauty of mathematics is that there are many different ways of looking at the same object, so physicists can deal with their arrays of numbers, I can play with my multilinear functions, and everyone gets the same answer when we ask the same question.

cygx 12y ago
I really dislike the "collection of numbers that transforms in this way" definition of vectors, tensors, etc. It is not totally without merit if you consider the tensor bundle as associated to the principal bundle of linear frames. However, that's hardly suitable as an introduction to the topic.
- tel 12y ago
  It's probably the Wikipedia introduction, though...

Enzolangellotti 12y ago

So, a tensor is kind of a Rubik Cube shaped matrix?

ajkjk 12y ago
A rank-3 tensor is. It's any number of dimensions you need.