3 ms·
Transformers can easily be trained / designed to handle grids, it's just that off the shelf standard LLMs haven't been particularly, (although they would have s
by froobius 1y ago
Transformers can easily be trained / designed to handle grids, it's just that off the shelf standard LLMs haven't been particularly, (although they would have seen some)
- nine_k 1y agoAre there some well-known examples of success in it?
- thethimble 1y agoVision transformers effectively encode a grid of pixel patches. It’s ultimately a matter of ensuring the position encoding incorporates both X and Y and position. For LLMs we only have one axis of position and - more importantly - the vast majority of training data only is oriented in this way.