The most likely "real solution" is going to be using various tricks and finetuning on higher context lengths to just extend the context window. However I will s

The most likely "real solution" is going to be using various tricks and finetuning on higher context lengths to just extend the context window. However I will say knowledge graphs are becoming more popular. I made a comment the other day with a list of some of the popular methods: https://news.ycombinator.com/item?id=38476596 https://news.ycombinator.com/item?id=38476596 I completely forgot to mention ROME in my last comment, where you can modify facts within an LLM https://arxiv.org/pdf/2202.05262.pdf https://arxiv.org/pdf/2202.05262.pdf Note that there could be something more recent then these that I missed, but as far as I know knowledge-graphs/RAG are what most people are currently using, but there's a lot of work being focused on extending the context window.