3 ms·
And AlphaGo. I especially liked "Decoupled Neural Interfaces using Synthetic Gradients" because it seemed to open up training of complex models on multiple mach
by visarga 10y ago
And AlphaGo. I especially liked "Decoupled Neural Interfaces using Synthetic Gradients" because it seemed to open up training of complex models on multiple machines, but also because it provided an unexpected insight into how local learning can be done.
- gwern 10y agoI'm still waiting to see synthetic gradients get applied anywhere else. When it came out, I thought it would at the very least revolutionize training of RNNs by giving them better BPTT, but it seems to've sunk without a trace.