3 ms·
Naive question: what's the intuition for how this is different from increasing the number of learnable parameters on a regular MLP?
by zygy 2y ago
Naive question: what's the intuition for how this is different from increasing the number of learnable parameters on a regular MLP?
- slashdave 2y agoOrthogonality ensures that each weight has its own, individual importance. In a regular MLP, the weights are naturally correlated.