3 ms·
What I mean is if context length wasn't a limit, you could have your own AI agent that remembers that article forever, along with everything else you've ever di
by chpatrick 3y ago
What I mean is if context length wasn't a limit, you could have your own AI agent that remembers that article forever, along with everything else you've ever discussed with it.
If it generated text on its own, without human interaction, and developed its "personality", then you could say it's more like what we consider an intelligent being.
- mr_toad 3y agoContext length is limited by the length of articles in the training data. You couldn’t use infinite context lengths unless the LLM had been trained on infinity long articles. Context lengths aren’t arbitrary limits - they reflect the ability (and limits) of the model to actually reason about lengthy topics. If you want longer context lengths you’ll need more training data, larger models, and a lot more money.
- consumer451 3y agoThank you. So in my very uniformed mind.. "AGI," at the very least, would require real-time retraining... correct?
- taf2 3y agoIMO yes, but more likely and entirely different yet to be discovered way of modeling intelligence