The trick is that the rules it follows aren't the ones people write. The real ones just happen to give similar answers, until one day they don't. The LLM takes

The trick is that the rules it follows aren't the ones people write. The real ones just happen to give similar answers, until one day they don't. The LLM takes a document and returns a "fitting" token that would go next. So "Calculate 2+2" may yield a "4", but the reason it gets there is document-fitting, rather than math.