2 ms·
I'm not sure they've written that anywhere else (which is a cautionary lesson for anyone trying to understand how research happens, BTW), but as further evidenc
by gwern 2y ago
I'm not sure they've written that anywhere else (which is a cautionary lesson for anyone trying to understand how research happens, BTW), but as further evidence besides just 'Ethan told me so on EAI Discord [IIRC]', you can see that in the original Reddit discussion where I mention Caballero's poster conversation, the lead author comments several times and doesn't contradict that anecdote: https://www.reddit.com/r/mlscaling/comments/n78584/grokking_generalization_beyond_overfitting_on/ https://www.reddit.com/r/mlscaling/comments/n78584/grokking_...