3 ms·
Grokking is fascinating! It seems tied to how neural networks hit critical points in generalization. Could this concept also enhance efficiency in models dealin
by alizaid 2y ago
Grokking is fascinating! It seems tied to how neural networks hit critical points in generalization. Could this concept also enhance efficiency in models dealing with non-linearly separable data?
- wslh 2y agoCould you expand about grokking [1]? I superficially understand what it means but it seems more important that the article conveys. Particularly: > Grokking can be understood as a phase transition during the training process. While grokking has been thought of as largely a phenomenon of relatively shallow models, grokking has been observed in deep neural networks and non-neural models and is the subject of active research. Does that paper add more insights? [1] https://en.wikipedia.org/wiki/Grokking_(machine_learning)?wprov=sfti1 https://en.wikipedia.org/wiki/Grokking_(machine_learning)?wp...
- tanananinena 2y agoThis is probably the most interesting (and insightful) paper on grokking I’ve read recently: https://arxiv.org/abs/2402.15555 https://arxiv.org/abs/2402.15555