3 ms·
> EDIT: This last part isn't true. I think they are only looking at the intermediate layer of the FFN which does have a privileged basis. it does?
by whimsicalism 4y ago
> EDIT: This last part isn't true. I think they are only looking at the intermediate layer of the FFN which does have a privileged basis.
it does?
- sebzim4500 4y agoYeah, that's where they apply the activation function and that happens per neuron so you can't rotate everything and expect the same result.