4 ms·
I ain't saying that deep learning is unrelated to tensor algebra. E.g. low-rank approximation of matrix can be reformulated as a tensor problem. Apparently, the
by The_suffocated 8y ago
I ain't saying that deep learning is unrelated to tensor algebra. E.g. low-rank approximation of matrix can be reformulated as a tensor problem. Apparently, the following discussion in CrossValidated suggests that some deep learning researchers do try to do something clever using true tensors:
https://stats.stackexchange.com/questions/198061/why-the-sudden-fascination-with-tensors https://stats.stackexchange.com/questions/198061/why-the-sud...
But my point is that the current "tensor" nomenclature in deep learning software has little to do with tensors in physics or mathematics. The name is just confusing if not misleading.
- jhanschoo 8y agoWhat uses of MDArrays in deep learning don't correspond to tensor manipulations? Many aspects of deep learning theory are discussed in a coordinate-free manner as tensors or multilinear maps, with the MDArray simply referring to the coordinate representation. The operations on the MDArray then refer to a change in representation of the underlying linear-algebraic object, or a discretized version of a linear-algebraic operation on the underlying object. Hence I find the naming entirely appropriate.
- The_suffocated 8y agoWhat uses of MDArrays in deep learning don't correspond to tensor manipulations? That's news to me. All genuinely tensor-related operations in machine learning that I was aware of were about dimension reduction (tensor decompositions or low-rank tensor approximations). Maybe things have changed and I stand corrected.
- nabla9 8y agoRemember that NNs do inference forward but learning happens backwards.