2 ms·
Longer context means more memory, effectively a longer history the LLM remembers. One issue i'm having is say functions works wonderfully, but context window is
by seanthemon 3y ago
Longer context means more memory, effectively a longer history the LLM remembers. One issue i'm having is say functions works wonderfully, but context window is tight even with 16k tokens, with a bigger context, sky is the limit.
- stevemadere 3y agoSo specifically, you are saying that LLM coding assistant currently gets confused when working on a large source file but if it had room for more context, you could get better help in writing code because it would have understanding of the entire module. Correct?
- seanthemon 3y agoTechnically so, but not confused - it may hallucinate but that's unrelated to context window (and more due to dataset and number of parameters) However, a longer context does mean you can give a lot more verbose instruction, maybe feed a whole code base over just a single script to give wider context over the current problem