Thanks, this is how I view it, too: there will always be relevant context that was unavailable at training, eg because it didn't exist yet, because it doesn't b

Thanks, this is how I view it, too: there will always be relevant context that was unavailable at training, eg because it didn't exist yet, because it doesn't belong to the trainers, or because it wasn't yet known to be relevant. One of these will remain true until every person has their own pet model which is fine-tuned, on keyup, on all public data and their own personal data. Still, something heinously parametric (like regional weather on some arbitrary date) I struggle to imagine fitting into a transformer. Edit: I can imagine every user getting a LoRA.