2 ms·
Not about that quote in particular, but the original grokking paper from Powers et al came about because they accidentally left a training job to run too long (
by scarmig 2y ago
Not about that quote in particular, but the original grokking paper from Powers et al came about because they accidentally left a training job to run too long (at least as related by gwern).
- gwern 2y agoI'm not sure they've written that anywhere else (which is a cautionary lesson for anyone trying to understand how research happens, BTW), but as further evidence besides just 'Ethan told me so on EAI Discord [IIRC]', you can see that in the original Reddit discussion where I mention Caballero's poster conversation, the lead author comments several times and doesn't contradict that anecdote: https://www.reddit.com/r/mlscaling/comments/n78584/grokking_generalization_beyond_overfitting_on/ https://www.reddit.com/r/mlscaling/comments/n78584/grokking_...