4 ms·
Transformers (GPT, etc.) cannot generalize beyond their training data
- thelastparadise 3y agoThis seems like a "well, duh" type of result. But there's other factors at play. There's an evolution of LLMs happening right now, which is very similar to what happened with stable diffusion. Specifically, there are publicly available LLM weights available. Many disparate, independent individuals and organizations are fine-tuning these models, creating LoRAs, and new foundational models. These weights are being merged, and further fine-tuned. This creates a system where humans are "evolving" the LLMs such that they change and adapt to requirements. And, of course, we select only for the best performing LLMs. Due to this, GPT-3 has been surpassed by open models, and we have specialist models that are better than GPT-4 in their domains. It seems to be only a matter of time that open LLMs are better than GPT-4. So, on their own, LLMs may not be able to generalize beyond their training set. But from a higher level, there is an ecosystem of evolving LLMs which do generalize (and sometimes specialize). As far as I'm aware, the fine tuning/creating LoRAs on new datasets that humans are doing could also be done in an automated fashion. So you'd need a system which accepts new external data/inputs, fine tunes on those, and some evaluation metric to choose the best LLMs. With such a system, it seems possible that LLMs could automatically learn and adapt to new information, much how humans do.
- RugnirViking 3y agothis is talking about "in-context" learning, i.e. you tell the bot some new information (here is a language, here are its rules, now translate this text) If it hasn't seen a language with those rules before, it makes a lot of mistakes. The key is with those sorts of rules. If it's seen one with similar rules but ordered differently, it will do much better than with entirely novel rules. I imagine a human would too. The paper itself trained a transformer to predict what function some data points represented, either linear (x=y or similar) or sinusoid (y=sin(x) or similar). They then presented a section of the sinoid data to the transformer, so the correct output would have been something similar to the sin function. they repeated this with models trained on various mixtures of linear and sinoid distributions, making the sinoids increasingly rarer and eventually not appearing in the distribution at all. Predictably, the accuracy got worse as the model saw less of the sinoids. You may note that these are vastly less sophisticated models than GPT.