The description of the attention mechanism of GPT architectures and a couple of examples can be very brief. Then you have to supply your imagination and realize

The description of the attention mechanism of GPT architectures and a couple of examples can be very brief. Then you have to supply your imagination and realize that the model simply found a whole bunch of very effective such attention measures by itself, which are all computed for every query, and they could be anything from the straight forward examples, or some more clever abstract types of attention. I think we'll need some more descriptions of what the more important attention measures it comes up with really are, to understand better.