Would attention sinks work here? https://github.com/mit-han-lab/streaming-llm https://github.com/mit-han-lab/streaming-llm - it sounds like they might. In the

Would attention sinks work here? https://github.com/mit-han-lab/streaming-llm https://github.com/mit-han-lab/streaming-llm - it sounds like they might. In theory it doesn't involve retraining, it's just a change to how the data is managed between invocations.