3 ms·
The most abstract mathematical definition of tensor I know of is multilinear map of some kind. The tensor data structures are of course just multidimensional
by nabla9 8y ago
The most abstract mathematical definition of tensor I know of is multilinear map of some kind.
The tensor data structures are of course just multidimensional arrays, but so are tensors in physics computations. Complex numbers are usually 2-dimesional arrays just like two dimensional vectors. It's the extra structure and operations that defines what they are.
I think the tensor in tensor flow comes from Tensor networks theory from 80's. https://en.wikipedia.org/wiki/Tensor_network_theory https://en.wikipedia.org/wiki/Tensor_network_theory
- The_suffocated 8y agoI ain't saying that deep learning is unrelated to tensor algebra. E.g. low-rank approximation of matrix can be reformulated as a tensor problem. Apparently, the following discussion in CrossValidated suggests that some deep learning researchers do try to do something clever using true tensors: https://stats.stackexchange.com/questions/198061/why-the-sudden-fascination-with-tensors https://stats.stackexchange.com/questions/198061/why-the-sud... But my point is that the current "tensor" nomenclature in deep learning software has little to do with tensors in physics or mathematics. The name is just confusing if not misleading.
- jhanschoo 8y agoWhat uses of MDArrays in deep learning don't correspond to tensor manipulations? Many aspects of deep learning theory are discussed in a coordinate-free manner as tensors or multilinear maps, with the MDArray simply referring to the coordinate representation. The operations on the MDArray then refer to a change in representation of the underlying linear-algebraic object, or a discretized version of a linear-algebraic operation on the underlying object. Hence I find the naming entirely appropriate.
- The_suffocated 8y agoWhat uses of MDArrays in deep learning don't correspond to tensor manipulations? That's news to me. All genuinely tensor-related operations in machine learning that I was aware of were about dimension reduction (tensor decompositions or low-rank tensor approximations). Maybe things have changed and I stand corrected.
- nabla9 8y agoRemember that NNs do inference forward but learning happens backwards.