3 ms·
Historically ML frameworks have only supported simple combinations of matrix multiply and convolution; it's really about generalising the idea of a model to any
by one-more-minute 8y ago
Historically ML frameworks have only supported simple combinations of matrix multiply and convolution; it's really about generalising the idea of a model to anything that can mathematically be differentiated [1]. For example, an ODE solver is differentiable and we can throw it into the forward pass of a neural net [2].
People have done this kind of thing before, but only by e.g. reimplementing a very limited physics engine in Theano. We can just reuse existing code, along with the significant domain expertise embedded in it, and make this kind of thing a ten-line script.
[1] https://fluxml.ai/2019/02/07/what-is-differentiable-programming.html https://fluxml.ai/2019/02/07/what-is-differentiable-programm...
[2] https://julialang.org/blog/2019/01/fluxdiffeq https://julialang.org/blog/2019/01/fluxdiffeq
- cs702 8y agoMike: THANK YOU for doing this work and sharing it with the world. It's awesome. Because of your work, I've started experimenting with Julia for building experimental DL/ML models, instead of Python+PyTorch, which is the predominant stack we use at work today for iterative experimentation. (For those here who don't know, one-more-minute is Mike Innes, the author of the paper.)
- one-more-minute 8y agoThanks for the kind words! I'm lucky to be part of such a smart and driven community – there's a really strong shared sense of how ML should look in Julia. I'm always happy to hear about how this stuff is being used, if people want to reach out.
- b_tterc_p 8y agoThank you for this. These articles are exceptionally clear and it looks like you’re the author. Fantastic work.