5 ms·
This helps. Thank you. Any advice on where to look to understand why the word tensor was used?
by Waterluvian 4y ago
This helps. Thank you. Any advice on where to look to understand why the word tensor was used?
- deleted 4y ago[deleted]
- avereveard 4y agoTo reduce it a little, matrix holds numbers, tensor holds whatever. Numbers. Vectors. Operations.
- yreg 4y agoSo it is math lingo for array.
- squeaky-clean 4y agoIn math tensor has a more specific meaning than just an array, but in machine learning lingo, tensor just means an array or multidimensional array. That base is true in math as well, but it can also be a single value (rank zero tensor, aka a scalar. Sometimes ML will call this a tensor as well) and has to obey certain rules about linearity for some operations applied to it.
- sva_ 4y agoIt should be noted that it doesn't have all to much to do with the rigorous mathematical definition of a Tensor.
- jayalammar 4y agoI updated the post to say "multi-dimensional array". In a context like this, we use tensor because it allows for any number of dimensions (while vector/ array is only one, matrix is two). When you get into ML libraries, both popular packages PyTorch and TensorFlow use the "tensor" terminology. It's a good point. Hope it's clearer for devs with "array" terminology.
- zestyping 4y ago> we use tensor because it allows for any number of dimensions "Vector" implies one dimension and "matrix" strongly implies two. But an array can have any number of dimensions, so "array" is the best word. We don't need the word "tensor"; when the context is programming, "tensor" is only confusing and doesn't really add any useful meaning.
- OkayPhysicist 4y agoTensor does imply a set of operations that are expected. Multiplying two arrays together is an ambiguous operation; multiplying two tensors together is well-defined. And really, the context is math, not programming. The programming side of DL is approximately trivial, the interesting bits are all represented as mathematical operations on a set of tensors.
- paulgrant999 4y ago
- leereeves 4y agoIt's especially confusing as math, because the standard meaning of "tensor" in math includes a lot of baggage that's not used in deep learning (related to physics and differential geometry).
- zestyping 4y agoExactly.
- kgwgk 4y agoInterestingly the Wikipedia page Tensor (disambiguation) https://en.wikipedia.org/wiki/Tensor_(disambiguation) https://en.wikipedia.org/wiki/Tensor_(disambiguation) has an entry Tensor type (computing) which is just a redirection to Array (data type) https://en.wikipedia.org/wiki/Array_(data_type) https://en.wikipedia.org/wiki/Array_(data_type) Doesn’t seem to support the relevance of “tensor” as something distinct from “array”.
- pvarangot 4y agoA Tensor is a mathematical object for symbolic manipulation of relationships between other objects that belong in conceptually similar universes or spaces. Literature on Deep Learning, like Goodfellow, call for the CS-minded reader to just assume it's a fancy word for a matrix of more than two dimensions. That makes matters confusing because mathematically you could have scalar or vectorial tensors. The classic mathematical definition puts more restrictions on the "shape" of the "matrix" by requiring certain properties so that it can create a symbolic language for a tensor calculus. Then you can study how the relationships change as variables on the universes or spaces change. Understanding the inertia tensor on classical mechanics or the stress tensor may illustrate where tensors come from, and I understand that GR also makes use of a lot of tensor calculus that came to be as mathematics developed manipulating and talking about tensors. I have a kinda firm grasp on some very rudimentary tensor calculus from trying to leanr GR, and a pretty solid grasp on classical mechanics. I've had hour long conversations with deep learning popes and thought leaders in varying states of mind and after that my understanding is that they use the word tensor in an overreaching fashion as like you could call a solar panel a nuclear fission reactor power source. This thought leaders include people with books and 1M+ view Youtube videos on the subject that use the word tensor and I'm not saying their names because they off-the-record admitted that it's a poor choice of term but it harms the ego of many Google engineers to publicly admit that.
- amelius 4y agoHow are sum-types handled in deep learning? E.g., a type that holds "a 3x2 tensor OR a 4x6x2x5 tensor".
- nicd 4y agoI'm not aware of tensor shape type safety (in general) being widely used in deep learning, let alone sum-types. I believe Pytorch and TensorFlow lack support for tensor shape type checking (looks like there is a Pytorch issue open: https://github.com/pytorch/pytorch/issues/26889 https://github.com/pytorch/pytorch/issues/26889).
- 4y ago
- jamessb 4y agoVery loosely, a number/vector/matrix/tensor can be considered to be objects where specifying the values of 0/1/2/3 indexes will give a number. (A mathematician might object to this on several grounds, such as that vectors/matrices/tensors are geometric objects which need not be expressed numerically as coordinates in any coordinate system)
- amelius 4y agoA tensor can take any number of dimensions (not just 3).
- kgwgk 4y agoIt’s a cool name from the good ol’ “multi-dimensional arrays with a uniform type” https://www.tensorflow.org/guide/tensor https://www.tensorflow.org/guide/tensor