3 ms·
i don't think grokking has been demonstrated in a large model yet
by whimsicalism 2y ago
i don't think grokking has been demonstrated in a large model yet
- HarHarVeryFunny 2y agoThere's a recent paper here where they've seen in a deeper model. https://arxiv.org/abs/2405.19454 https://arxiv.org/abs/2405.19454 It's a bit surprising that apparently this hasn't been done before - to see how universal of a phenomenon this is.