I'm OK with the term emergent used here - it seems the best word to describe non-trivial capabiities/properties that weren't designed in. At least prior to the

I'm OK with the term emergent used here - it seems the best word to describe non-trivial capabiities/properties that weren't designed in. At least prior to the first LLMs, I think most people would just expect them to be capable of doing literally what they are trained to do - predict (locally plausible) next word(s), and this is certainly all that is "designed in" if we're talking about a basic LLM (vs one with further RL-based "alignment", etc). Of course we can also appreciate that to get REALLY REALLY good at "predict next word" would require intelligence/understanding of what is being generated, but I think the point here is would anyone - the model designers in particular - have expected that the transformer architecture (+ scale) is all it would take to become so good at this task? I don't think "attention is all you need" was really anticipating transformers reading API docs and generating code to perform requested functions! One might have expected it to take a much more elaborate and evolved architecture to achieve this level of capability! So, to me, it seems entirely appropriate to describe these models as having emergent capabilities - things they are capable of doing that are really so far above and beyond "predict next word", that it seems churlish and inaccurate to describe them as designed it (or even just confidently predicted).