4 ms·
Differentiable programming for gradient-based machine learning
- dash2 6y agoHas anyone got experience in using other things than python for machine learning? What are the rivals out there?
- gumby 6y agoIn my experience: Julia, Java, C++. Lack of libraries in a pain; OTOH the quality of most libraries (in general) tends to be poor. Examples of library issues: there is C++ support in tensor flow, but not for training.
- undefinednull 6y agoAgree to that completely. It is also hard for the libraries to spend their energy / time in building for languages that covers only a few percentage of total users. :(
- oscardssmith 6y agoFor anything involving differential equations, Julia is way better than anything else. I also wouldn't be at all surprised if Julia catches up to python for way more machine learning within the next year
- dunefox 6y agoJulia doesn't lack libraries and the quality imo is pretty good. Did you look at PyCall.jl?
- undefinednull 6y agoInteresting! I did not yet! will take a look. I remember swift was working on something similar to seamlessly call Python.
- jrumbut 6y agoParticularly outside deep learning, R remains popular especially for those closer to the stats world. Matlab and Mathematica both have interesting machine learning features but are probably orders of magnitude lower usage levels compared to Python.
- dunefox 6y agoI'm using Julia instead of Python for NLP and it's great. If I need a python library I just use PyCall and so far it has worked flawlessly.
- dash2 6y agoCan you say a bit about the advantages of each? I'm just starting to learn deep learning and planning to use it for NLP. I'm taking the python-based fast.ai course. But I'm tempted by the elegance of Julia.
- dunefox 6y agoIf you're starting you should use Python and when you have a more solid grasp you can switch to Julia. That's mostly due to the fact that there are much more tutorials and resources for dl in python.
- samuell 6y agoThere is a rather active community for Go in data science and machine learning. +1700 members on the #data-science channel on the Gopher slack, a few books on the topic, and a few libraries: Books Machine learning with Go - https://www.amazon.com/Machine-Learning-Classification-Time-Programming-ebook/dp/B01LPRN11G https://www.amazon.com/Machine-Learning-Classification-Time-... Go Machine Learning Projects - https://www.amazon.com/Go-Machine-Learning-Projects-Ultimate-ebook/dp/B07D6L111J/ https://www.amazon.com/Go-Machine-Learning-Projects-Ultimate... Libraries https://gorgonia.org/ https://gorgonia.org/ https://www.gonum.org/ https://www.gonum.org/ A good talk (although from a from years back), explaining why Go for ML could be a good idea: https://www.youtube.com/watch?v=D5tDubyXLrQ https://www.youtube.com/watch?v=D5tDubyXLrQ
- sfletcher 6y agoIs anyone still excited about differentiable programming? At least in NLP it seems like a lot of the energy has shifted to large scale overparameterized models like BERT, e.g. you don't need arbitrary control flow in your model, all you need is attention.
- KenoFischer 6y agoThe differentiable programming framing is quite useful in the scientific simulation space, because it lets you couple traditional physics simulations to either neural surrogates or some sort of bayesian model. Enough of it is happening in the Julia world that were very actively improving the compiler to support it.
- pixelpoet 6y agoHere's a stunning recent example from computer graphics: http://rgl.epfl.ch/publications/NimierDavidVicini2019Mitsuba2 http://rgl.epfl.ch/publications/NimierDavidVicini2019Mitsuba...
- bionhoward 6y agoAttention is cool but not to the level of higher-order cognitive architecture. Further, there are many uses of differentiable programming besides AI
- samuell 6y agoThis talk by Eric Meijer got me intrigued of the subject ... (Based on this talk, I think I almost understood it for an hour or two): "The principles behind Differentiable Programming - Erik Meijer" https://www.youtube.com/watch?v=lk0PhtSHE38 https://www.youtube.com/watch?v=lk0PhtSHE38