3 ms·
No mention of the PixelCNN WaveNet VPN ByteNet line of research?
by rough-sea 10y ago
No mention of the PixelCNN WaveNet VPN ByteNet line of research?
- visarga 10y agoAnd AlphaGo. I especially liked "Decoupled Neural Interfaces using Synthetic Gradients" because it seemed to open up training of complex models on multiple machines, but also because it provided an unexpected insight into how local learning can be done.
- gwern 10y agoI'm still waiting to see synthetic gradients get applied anywhere else. When it came out, I thought it would at the very least revolutionize training of RNNs by giving them better BPTT, but it seems to've sunk without a trace.