3 ms·
grokking doesn't and will not have practical uses, imo - it is just an experiment that revealed cool things that we mostly already suspected about implicit regu
by whimsicalism 2y ago
grokking doesn't and will not have practical uses, imo - it is just an experiment that revealed cool things that we mostly already suspected about implicit regularization
however, techniques we learn from grokking about implicit regularization might be helpful for the training regimes we actually use
- naasking 2y ago> grokking doesn't and will not have practical uses, imo I'm not so sure. Reasoning is the next big hurdle, and grokking and parametric memory seem very effective here. [1] https://arxiv.org/abs/2405.15071 https://arxiv.org/abs/2405.15071