3 ms·
Since I first learned about grokking, I've had a strong suspicion that getting a handle on it and figuring out how to aid it should be the central question in A
by scarmig 2y ago
Since I first learned about grokking, I've had a strong suspicion that getting a handle on it and figuring out how to aid it should be the central question in AI. We are currently stuck in a local minimum, where memorizing circuits perform well-enough to handle a whole lot of economically viable use cases. But the profit function has guided us into a valley dominated by a data and compute hungry architecture that isn't ideal for learning generalizing circuits (partially because the memorizing circuits are so effective! We relatively quickly get to a flat loss landscape, after which we blindly jump around for countless epochs in a kind of Brownian motion until we get into an area where regularizers can drive generalization). Research like this paper is incredibly important.
I thought this was the most interesting bit from the paper:
> Training data distribution, instead of training data size, qualitatively influences generalization behavior.
- aantix 2y agoIf there are many examples of X on the web, and only a few examples of related Y, would you actively prohibit the additional X samples to be included in the training set to prevent the reinforcement of the common case?
- scarmig 2y agoFor the purpose of generalization, I think that's more treating a symptom than the root cause. The better approach IMO would be finding architectures that heavily penalize the formation of memorizing and interpolating circuits. E.g much stronger weight decay than used today.
- taskforcegemini 2y agoI'm sure this is what happened to google search
- scarmig 2y agoKind of, but that's not inherent to the over reliance on memorization. I suspect using the top tier models from any of OAI, Anthropic, or Google would have resulted in much less embarrassing results, and I believe they're all primarily memorizers, not generalizers. The search issue happened because Google had to use a really cheap model to power the search results, and a memorizing model that cheap is going to be highly constrained in capabilities (at least right now).
- chaorace 2y agoIt doesn't help that they were caught with their pants down back when GPT-3 first entered the zeitgeist (and thus also the radar of Google's institutional stakeholders). Haste made waste, corners got cut, cargo went overboard.