3 ms·
I would love for a non-python based deep learning framework to gain traction. My initial impression though is that the scope is very broad. Trying to be both s
by angusturner 3y ago
I would love for a non-python based deep learning framework to gain traction.
My initial impression though is that the scope is very broad. Trying to be both sci-kit learn and numpy and torch seems like a recipe for doing none of these things very well.
Its interesting to contrast this with the visions/aspirations of other new-ish deep learning frameworks. Starting with my favorite, Jax offers "composable function transformations + autodiff". Obviously there is still a tonne of work to do this well, support multiple accelerators etc. etc. But notably I think they made the right call to leave high level abstractions (like fully-fledged NN libraries or optimisation libraries) out of the Jax core. It does what it says on the box. And it does it really really well.
TinyGrad seems like another interesting case study, in the sense that it is aggressively pushing to reduce complexity and LOC while still providing the relevant abstractions to do ML on multiple accelerators. It is quite young still, and I have my doubts about how much traction it will gain. Still a cool project though, and I like to see people pushing in this direction.
PyTorch obviously still has a tonne of mind-share (and I love it), but it is interesting to see the complexity of that project grow beyond what it is arguably necessary. (e.g. having a "MultiHeadAttention" implementation in PyTorch is a mistake in my opinion).
- ezquerra 3y agoI am one of the Arraymancer contributors. I believe that what mratsim (Arraymancer’s creator) has done is pretty amazing but I agree that the scope is a quite ambitious. There’s been some talk about separating the deep learning bits into its own library (which I expect would be done in a backwards compatible way). Recently we worked on adding FFT support but instead of adding it to Arraymancer it was added to “impulse” (https://github.com/SciNim/impulse https://github.com/SciNim/impulse) which is a separate, signal processing focused library. There is also Vindaar’s datamancer (a pandas like dataframe library) and ggplotnim (a plotting library inspired by R’s ggplot). The combination of all of these libraries makes nim a very compelling language for signal processing, data science and ML. Personally I’d like Arraymancer to be a great tensor library (basically a very good and ideally faster alternative to numpy and base Matlab). Frankly I think that it’s nearly there already. I’ve been using Arraymancer to port a 5G physical layer simulator from Matlab to nim and it’s been a joy. It’s not perfect by any means but it’s already very good. And given how fast nim’s scientific ecosystem keeps improving it will only get much better.
- pjmlp 3y agoI see Julia and eventually Mojo gaining more adoption than anything else, this without taking into account that finally JIT efforts have started to be taken more seriously by the community after PyPy feeling quixotic for so many years. There are also the JIT GPU efforts from Intel and NVIDIA into their APIs. Personally I would like to see more Java and .NET love, however dynamic languages loved by the research community is where the game is at, also the reasoning behind Mojo, after the Swift for Tensorflow failure. Naturally kudos to the Arraymancer effort, the more the better.
- machinekob 3y agoMojo is AOT not JIT or is there are some sort JIT in mojo? (I'm not up to date with new development there) I feel like JIT is just kinda bad for Deep learning in most cases you want to "compile and optimise" your graph before run and load it fast for running/training.
- gcr 3y agoJax is so complex though! Autograd using bytecode reflection to inspect CPython’s interpreter state to emit Jax’s own front end ILR (jaxpr), the Jax-specific compiler (XLA) that lowers HLO down to at least three different implementation backends (cpu, TPU, GPU for CUDA, maybe more…) Then there’s the JIT that Jax also brings to the table. All of that to make something that seems simple on the surface. You could say that Jax is simultaneously trying to be numpy, Theano/sympy, PyPy/numba, and pyCUDA all at the same time. Both systems are trying to be so much. Perhaps the difference is Jax’s focus on a narrower developer interface.