4 ms·
I think most ML models aren’t very “lean”, meaning there is space in their weight layers for information isn’t directly attributable to predictive accuracy. Tha
by rustyconover 7y ago
I think most ML models aren’t very “lean”, meaning there is space in their weight layers for information isn’t directly attributable to predictive accuracy. That space is likely where this new “radioactive” like data is being “stored”/“remembered”.
The leanness could be increased during training by progressively trimming width/depth of weights, but I doubt if every model has this done.
- lmkg 7y agoThis is definitely true. In fact, this can be exploited to extract sensitive/private attributes about the training data from the learned models. This may become an issue for, e.g., AI in healthcare. "Overlearning Reveals Sensitive Attributes": https://arxiv.org/abs/1905.11742 https://arxiv.org/abs/1905.11742