This is a very useful answer, thanks. So is every possible 16x16 grid of pixels a legal token value and they are used more or less verbatim or are they encoded

This is a very useful answer, thanks. So is every possible 16x16 grid of pixels a legal token value and they are used more or less verbatim or are they encoded in some other way? Prompted by some of the other replies I’ve read up a bit on positional embeddings. That they are used even with text transformers which otherwise lack the linearity of RNNs etc helps tremendously to clarify things.