I've been wondering about how to do incremental updates without incurring the cost of a full recalculation of the training data. I suppose I assumed that LLM pr

I've been wondering about how to do incremental updates without incurring the cost of a full recalculation of the training data. I suppose I assumed that LLM providers would (if not now, eventually) incorporate a fine-tuning step to update a model's self-knowledge before making the model available. This would avoid including the update in the context length. Among many, many applications, this would be helpful in allowing LLMs to converse about the current version of a website or application. I'd want a sense of time to be maintained, so that the LLM would know, if asked, about various versions. "Before the April 5, 2023 update, this feature was limited to ..., but now ... is supported." I asked GPT4 about incremental updates, and it seemed to validate by my basic understanding. Here's the conversation so far: https://chat.openai.com/share/00fe148a-13aa-4e92-8b77-f0de486c6522 https://chat.openai.com/share/00fe148a-13aa-4e92-8b77-f0de48...