2 ms·
Not really. A model (after training) continues to be as plastic as it was before. Correspondingly, it can be made to "forget" what it has "learned" and learn so
by ssivark 7y ago
Not really. A model (after training) continues to be as plastic as it was before. Correspondingly, it can be made to "forget" what it has "learned" and learn something else. [At least architecture-wise. Whether the learned parameter values reduces plasticity is a non-trivial statement that needs to be demonstrated]
If contemporary AI/ML models behaved like what Alison Gopnik says, then their tendency to overfit would be even more of a problem -- you couldn't even transfer them from simulation domain to reality -- since they would lose all their plasticity overfitting to simulation!
Also, this article contains lots of other interesting ideas to think about. Highly recommend reading all of it.
- personjerry 7y agoThe learning coefficient decreases over time making the model less plastic.
- vankessel 7y agoOnly because you have chosen for the learning rate to decrease over time so the model converges. You could reset it. I would even argue the LR is not relevant to plasticity, it's a meta-variable for training the model and not to do with the model itself. A plastic model would be one that could accomplish a given task after being trained on it, and then trained on a second task while maintaining the ability to accomplish the first task.