4 ms·
Could you quantify it (Twice, three times, etc)? Another question, does it apply to only attention heads learning or the whole shebang?
by rnosov 4y ago
Could you quantify it (Twice, three times, etc)?
Another question, does it apply to only attention heads learning or the whole shebang?
- adamnemecek 4y agoIt's going to be more than that, like significantly. Hopf coherence theoretically makes away with backprop altogether. I'm in the process of implementing it but it might take sometime. I'm aware of Hinton's forward-forward, but I need more time to provide a comparison.