In-context learning (ICL) is a thing. People aren't entirely sure how it works[1]. LLMs are very effective at few-shot learning via so yes, for all practical p

In-context learning (ICL) is a thing. People aren't entirely sure how it works[1]. LLMs are very effective at few-shot learning via so yes, for all practical purposes yes, large context windows do allow for continuous learning. Note that the context needs to be loaded and processed on every request to the LLM though - so all that additional information has to be "retaught" each time. [1] https://openreview.net/pdf?id=992eLydH8G https://openreview.net/pdf?id=992eLydH8G "These results indicate that the equivalence between ICL (ed: In-context-learning) and GD (ed: Gradient Descent) is an open hypothesis, requires nuanced considerations, and calls for further studies.