5 ms·

Are these libraries ever useful in non-deep learning applications? It sounds like Theano is a bit more general purpose, but why would I ever need it outside of

Are these libraries ever useful in non-deep learning applications? It sounds like Theano is a bit more general purpose, but why would I ever need it outside of a deep learning context? I wonder if it could be used for something crazy, e.g. setting up a graph that generates shadertoy-like images on the GPU.

6gvONxR4sf7o 6y ago
They are. Lots of numerical code benefits from GPU and lots of numerical code benefits from derivatives. Simulations, solvers, numerical optimization, good old fashioned statistics.
timkpaine 6y ago
Idk about using these libraries, but its almost impossible to find generic graph libraries that aren't designed around either ML or alternatively scheduling batches. One such example is my own, https://github.com/timkpaine/tributary https://github.com/timkpaine/tributary
- nerdponx 6y ago
  Interesting library & idea, almost like its own programming paradigm when you abstract away all the specificity for building software or running ETL jobs or whatever. But this is a completely different kind of graph. The graphs being discussed here are differentiable DAGs of mathematical computations.
  - timkpaine 6y ago
    Is it that different? https://github.com/timkpaine/tributary/blob/main/docs/examples/autodiff/autodiff.md https://github.com/timkpaine/tributary/blob/main/docs/exampl...
    - nerdponx 6y ago
      Now that is definitely interesting. And you have some notion of "differentiability" for all of your various sources, sinks, and transforms? That said, Tensorflow and Pytorch are both very much general purpose numerical computing libraries. You don't have to use them for neural networks.
physicsyogi 6y ago
Libraries like this enable differentiable programming, which lets you backprop through more than just neural networks. For instance, people have built a differentiable raytracer and plugged a physics engine into reinforcement learning to accelerate training. https://en.wikipedia.org/wiki/Differentiable_programming https://en.wikipedia.org/wiki/Differentiable_programming
tcpekin 6y ago
We use them for computational imaging reconstruction in electron microscopy.

6gvONxR4sf7o 6y ago

They are. Lots of numerical code benefits from GPU and lots of numerical code benefits from derivatives. Simulations, solvers, numerical optimization, good old fashioned statistics.

timkpaine 6y ago

Idk about using these libraries, but its almost impossible to find generic graph libraries that aren't designed around either ML or alternatively scheduling batches. One such example is my own, https://github.com/timkpaine/tributary https://github.com/timkpaine/tributary

nerdponx 6y ago
Interesting library & idea, almost like its own programming paradigm when you abstract away all the specificity for building software or running ETL jobs or whatever. But this is a completely different kind of graph. The graphs being discussed here are differentiable DAGs of mathematical computations.
- timkpaine 6y ago
  Is it that different? https://github.com/timkpaine/tributary/blob/main/docs/examples/autodiff/autodiff.md https://github.com/timkpaine/tributary/blob/main/docs/exampl...
  - nerdponx 6y ago
    Now that is definitely interesting. And you have some notion of "differentiability" for all of your various sources, sinks, and transforms? That said, Tensorflow and Pytorch are both very much general purpose numerical computing libraries. You don't have to use them for neural networks.

physicsyogi 6y ago

Libraries like this enable differentiable programming, which lets you backprop through more than just neural networks. For instance, people have built a differentiable raytracer and plugged a physics engine into reinforcement learning to accelerate training. https://en.wikipedia.org/wiki/Differentiable_programming https://en.wikipedia.org/wiki/Differentiable_programming

tcpekin 6y ago

We use them for computational imaging reconstruction in electron microscopy.