Afaik the few-shots learning abilities of LLMs are much better than what you can achieve with fine tuning when you only have tiny samples. Of course it is true

Afaik the few-shots learning abilities of LLMs are much better than what you can achieve with fine tuning when you only have tiny samples. Of course it is true for real long context but it's not clear if its going to work with the sparse context hacks intended to keep memory size low. Having the ability to keep the same coding session forever without having the LLM forgetting all the time and making the same mistakes over and over would be a game changer.