6 ms·

We've got a winner. Tensors "do" something; they're functions. In the rank-1 and rank-2 case these functions can be represented as objects familiar to linear a

We've got a winner. Tensors "do" something; they're functions. In the rank-1 and rank-2 case these functions can be represented as objects familiar to linear algebra. 3-tensors are not the same as stacked two-tensors, however. I wonder if "neural networks as ODEs" will catch on. I could use some of my masters work and go get a PhD.

twtw 8y ago
Is it accurate to say that rank-n tensor is to n-D array of numbers as linear transformation is to matrix?
- krastanov 8y ago
  For a mathematician or a physicist, yes. But the word ended up having new, confusing meanings.
  - nabla9 8y ago
    It's the same for ML perspective as well.
joshuamorton 8y ago
>We've got a winner. Tensors "do" something; they're functions. And a neural network is just a stack of linear transforms (the "tensors") and nonlinear transforms (the activations). So in a lot of cases, these "tensors" do do something. Granted the abstraction for storing the active tensors is also a great abstraction for storing high-dimensional data (like a 128x128x4x60 2-second video). >neural networks as ODEs You're a few months late :P [0] [0]: https://arxiv.org/abs/1806.07366 https://arxiv.org/abs/1806.07366
Koshkin 8y ago
> they're functions Well, in math a lot of things we normally do not think of (or use) as a function can still be considered as such. For example, a vector "is a function." Why? Because it does something (to a covector, or a linear form - which, in turn, is a function, etc. etc.) Even a number "is a function" by a similar reasoning. Objects in math are very flexible conceptually.
- thanatropism 8y ago
  Ehhh. Take for simplicity a constant real number. A constant is not a function but there's a function from the reals to a space of real valued functions. Likewise every vector can be converted into a cove tot, but... You can say these function-valued functions are bijective so you can work on either side of the fence. Your can also work with equivalence classes if you don't quite have a bijection. But a lot of mathematics is about being specific about what we want, even if we're lazy in writing later and smush related concepts together in the same symbol.

twtw 8y ago

Is it accurate to say that rank-n tensor is to n-D array of numbers as linear transformation is to matrix?

krastanov 8y ago
For a mathematician or a physicist, yes. But the word ended up having new, confusing meanings.
- nabla9 8y ago
  It's the same for ML perspective as well.

joshuamorton 8y ago

>We've got a winner. Tensors "do" something; they're functions. And a neural network is just a stack of linear transforms (the "tensors") and nonlinear transforms (the activations). So in a lot of cases, these "tensors" do do something. Granted the abstraction for storing the active tensors is also a great abstraction for storing high-dimensional data (like a 128x128x4x60 2-second video). >neural networks as ODEs You're a few months late :P [0] [0]: https://arxiv.org/abs/1806.07366 https://arxiv.org/abs/1806.07366

Koshkin 8y ago

> they're functions Well, in math a lot of things we normally do not think of (or use) as a function can still be considered as such. For example, a vector "is a function." Why? Because it does something (to a covector, or a linear form - which, in turn, is a function, etc. etc.) Even a number "is a function" by a similar reasoning. Objects in math are very flexible conceptually.

thanatropism 8y ago
Ehhh. Take for simplicity a constant real number. A constant is not a function but there's a function from the reals to a space of real valued functions. Likewise every vector can be converted into a cove tot, but... You can say these function-valued functions are bijective so you can work on either side of the fence. Your can also work with equivalence classes if you don't quite have a bijection. But a lot of mathematics is about being specific about what we want, even if we're lazy in writing later and smush related concepts together in the same symbol.