> Where is the connection between computational details and the model's high-level behavior? I think for the most part we don't know. People at OpenAI/etc who

> Where is the connection between computational details and the model's high-level behavior? I think for the most part we don't know. People at OpenAI/etc who are training/testing these models and trying to control them no doubt have some understanding of how they are actually working, but they are certainly not claiming to fully understand. At a purely conceptual level I think the best way to begin to bridge the gap between plumbing and behavior is to forget the training objective and consider what the models must have been forced to learn in order to optimize that objective. Sutskever from OpenAI has called what they've learnt a "world model", meaning a model of the generative processes (the human mind and entities being discussed?) that are producing the sequence of words they are predicting. It's certainly way more abstract than learning some "stochastic parrot" surface level statistics of the training data, even if that's maybe a good starting point to describe it to a layman. It would be fascinating to know exactly how these models are performing reasoning - by analogy (abstract pattern matching) perhaps ?