Putting knowledge in the context window enables the same thing For example, letting the LLM return a result to the user and return a list of entities it wants

Putting knowledge in the context window enables the same thing For example, letting the LLM return a result to the user and return a list of entities it wants to save details on for later, then letting it query past entries allows long term memory for a class of entities. Over all it will be a very long time before approaches like that are beat by fine tuning, because as the models improve, this strategy gets stronger: the model gets better are crafting queries, you can provide more context in the context window. Meanwhile fine tuning in knowledge will get more difficult as MOE and other unforseen developments reset what works as far as directly tinkering with weights.