35 ms·
If you're looking for something to do, implement this: https://www.deepmind.com/blog/decoupled-neural-interfaces-using-synthetic-gradients https://www.deepmin
by throwawaymaths 3y ago
If you're looking for something to do,
implement this:
https://www.deepmind.com/blog/decoupled-neural-interfaces-using-synthetic-gradients https://www.deepmind.com/blog/decoupled-neural-interfaces-us...
in this:
https://hexdocs.pm/nx/Nx.html https://hexdocs.pm/nx/Nx.html
Someone at our local meetup said it was the future.
- bullfightonmars 3y agoWow. This paper is _very_ well written. I am not an expert in neural networks, but the concepts were very well described and diagramed and I was able to follow along. I would read a whole book about neural networks written in this style.
- cpursley 3y agoI’m experienced with Elixir but have no idea what all this is. Can you break it down for me and it’s benefits?
- throwawaymaths 3y agoNormally, ML training is via back propagation, which is a synchronous technique. If you try to trivially parallelize, it doesn't work, for reasons (tm). This lets you train a machine learning model of arbitrary size (bigger than can fit on a GPU, or even a multigpu node) using an actor-based distributed technique. There is a slight training cycles count penalty but it's way less than the cost of coordination.