Not exactly related, but I wonder sometimes if the fact that the weights in current models are very expansive to change is a feature and not a "bug". Somehow,

Not exactly related, but I wonder sometimes if the fact that the weights in current models are very expansive to change is a feature and not a "bug". Somehow, it feels harder to trust a model that could evolve over time. It's performance might even degrade. That's a steep price to pay for having memory built in and a (possibly) self-evolving model.