Small LMs do tend to avoid originality, but large LMs can use the context as a starting point to walk outside their training data distribution. They can do that

Small LMs do tend to avoid originality, but large LMs can use the context as a starting point to walk outside their training data distribution. They can do that by being better at composing concepts and using context-conditioning, which is also known as in-context learning.