3 ms·
It would be news is somebody showed transformers could generalize beyond the training data. Deep learning models generally cannot, so it's not a surprise this h
by andy99 3y ago
It would be news is somebody showed transformers could generalize beyond the training data. Deep learning models generally cannot, so it's not a surprise this holds for transformers.
- GaggiX 3y agoIt depends on what does "generalize beyond the training data" means. If I invent a new programming language and I teach (in-context) the language to the model and it's able to use it to solve many tasks, is it generalizing beyond the training data?
- andy99 3y agoNo. The way I'd look at it is that generalization or specifically extrapolation would mean that different features are needed to make a prediction (here, the next token) than what is seen in the training data. Something like a made up language could still result in the same patterns being relevant. That's why out-of-distribution research often uses mathematical extrapolation as a task.
- famouswaffles 3y ago"Supercharged Interpolation" is not something that actually exists. Learning in High Dimension Always Amounts to Extrapolation https://arxiv.org/abs/2110.09485 https://arxiv.org/abs/2110.09485 What you're asking for is not "generalization" but magic and humans would also fail.
- deleted 3y ago[deleted]
- spywaregorilla 3y agoCan you provide a real world example? Because this sounds like nonsense. As in, not a weakness of any architecture but just the very concept of pattern matching. What you might be asking for is a system that simply continually learns.
- vidarh 3y agoCan humans do this? Because I'm not convinced humans can do this. Or that it reasonably means anything.
- Swizec 3y ago> somebody showed transformers could generalize I read an interesting paper recently that had a great take on this: If you add enough data, nothing is outside training data. Thus solving the generalization problem. Wasn’t the main point of that paper, but it made me go ”Huh yeah … I guess … technically correct?”. It raises an interesting thought that yes if you just train your neural network on everything, then nothing falls outside its domain. Problem solved … now if only compute was cheap.
- andy99 3y agoCompute and data collection. This is the long tail problem.
- sharemywin 3y agodoesn't the long tail just need to be a little outside the range of human capability for it to be AGI?
- andy99 3y agoNot sure I understand but people don't need the long tail because we don't write rules and then blindly act on them when we encounter new things. We can reason about stuff we haven't seen before.
- cosmojg 3y agoHas it even been shown that the average human can generalize beyond their training data? Isn't this the central thrust of the controversy around IQ tests? For example, some argue that access to relevant training data is a greater determinant of performance on IQ tests than genetics[1]. [1] https://www.youtube.com/watch?v=FkKPsLxgpuY https://www.youtube.com/watch?v=FkKPsLxgpuY
- sudosysgen 3y agoJust because we can be genuinely creative in some way doesn't mean that having access to data doesn't make it easier.
- cosmojg 3y agoIndeed! And even then, can we be genuinely creative? As far as I can tell, everything is derivative.
- czzr 3y agoConsidering the human race bootstrapped itself from nothing, what’s your mental model here? Where did everything come from?
- cosmojg 3y agoRemembering the outcomes of accidents and remixing those memories to produce derivative works.
- kridsdale3 3y agoNature.
- throwaway092323 3y agoHumans and AIs both evolve as the result of some iterations dying. In both cases, we tacitly erase the ones who don't make it (by framing the discussion around the successful, alive ones). The difference is that humans have had a broader training set.
- YetAnotherNick 3y agoDefine "generalize beyond training data" please. Because in my opinion asking a model to produce avacado chair is generalizing beyond training data.
- huijzer 3y agoOpenAI showed it in 2017 with the sentiment neuron (https://openai.com/research/unsupervised-sentiment-neuron https://openai.com/research/unsupervised-sentiment-neuron). Basically, the model learned to classify the sentiment of a text which I would agree is a general principle, so the model learned a generalized representation based on the data. Having said that, the real question is what percentage of the learned representations do generalize. For a perfect model, it would learn only representations that generalize and none that overfit. But, that's unreasonable to expect for a machine *and* even for a human. Maybe we just don't know. We are staring at a black box and doing some statistical tests, but actually don't know whether the current AI architecture is capable enough to get to some kind of human intelligence equivalent.