3 ms·
Yes, that's the reason why I'm talking about it. It should be faster since it works locally (within the single layers) as opposed to across the whole graph.
by adamnemecek 4y ago
Yes, that's the reason why I'm talking about it. It should be faster since it works locally (within the single layers) as opposed to across the whole graph.
- rnosov 4y agoCould you quantify it (Twice, three times, etc)? Another question, does it apply to only attention heads learning or the whole shebang?
- adamnemecek 4y agoIt's going to be more than that, like significantly. Hopf coherence theoretically makes away with backprop altogether. I'm in the process of implementing it but it might take sometime. I'm aware of Hinton's forward-forward, but I need more time to provide a comparison.