4 ms·
Is anyone still excited about differentiable programming? At least in NLP it seems like a lot of the energy has shifted to large scale overparameterized models
by sfletcher 6y ago
Is anyone still excited about differentiable programming? At least in NLP it seems like a lot of the energy has shifted to large scale overparameterized models like BERT, e.g. you don't need arbitrary control flow in your model, all you need is attention.
- KenoFischer 6y agoThe differentiable programming framing is quite useful in the scientific simulation space, because it lets you couple traditional physics simulations to either neural surrogates or some sort of bayesian model. Enough of it is happening in the Julia world that were very actively improving the compiler to support it.
- pixelpoet 6y agoHere's a stunning recent example from computer graphics: http://rgl.epfl.ch/publications/NimierDavidVicini2019Mitsuba2 http://rgl.epfl.ch/publications/NimierDavidVicini2019Mitsuba...
- bionhoward 6y agoAttention is cool but not to the level of higher-order cognitive architecture. Further, there are many uses of differentiable programming besides AI