2 ms·
No it does not, not even a little bit: read my comment carefully instead of rushing to reply. > They cannot just inject "bits and pieces of newer information"
by BoorishBears 3y ago
No it does not, not even a little bit: read my comment carefully instead of rushing to reply.
> They cannot just inject "bits and pieces of newer information" to the old models as far as I know
Hence me saying:
>> the training cutoff date isn't some absolute date from which they never allowed any new data to be trained on
>> It's not going to reliably understand your new libraries, and more importantly if you convince it that it knows what happened in April 2023, it might start hallucinating extremely deeply
Fine tuning isn't for reliably adding information to a model, but you will end up adding novel sequences to the model that will very lossily capture new information.
It's more of an artifact of the process used than an intentional capability for end users to start trying to depend on. Once the context window contains enough "proof" that the model understands past it's cutoff, it will stop spitting out it's original disclaimer and seamlessly weave hallucinations into reality.