(Disclaimer: I am not an AI expert.) To me, LLMs are a special form "self-supervised learning"[1], which is a special form of unsupervised learning. While super

(Disclaimer: I am not an AI expert.) To me, LLMs are a special form "self-supervised learning"[1], which is a special form of unsupervised learning. While supervised learning aims to use input data to predict labels, self-supervised learning uses the context around the input data to predict future data. The key part here is context. This excellent article [2] by Yann Lecun et al. explains how self-supervised learning uses temporal and spatial context to extract patterns, which can then be used to predict future data or generate new data. In fact, prediction and generation go hand-in hand. So, you could definitely apply the techniques of LLMs to arbitrary streams of data with meaningful temporal or spatial patterns. [1] https://en.wikipedia.org/wiki/Self-supervised_learning https://en.wikipedia.org/wiki/Self-supervised_learning [2] https://ai.meta.com/blog/self-supervised-learning-the-dark-matter-of-intelligence/ https://ai.meta.com/blog/self-supervised-learning-the-dark-m...