3 ms·

It is hard to believe that modifying input datasets won't modify the qualitative behavior of the outputs in some way. This appears to be a modern variation of

• rustyconover 7y ago
  I think most ML models aren’t very “lean”, meaning there is space in their weight layers for information isn’t directly attributable to predictive accuracy. That space is likely where this new “radioactive” like data is being “stored”/“remembered”. The leanness could be increased during training by progressively trimming width/depth of weights, but I doubt if every model has this done.
  • lmkg 7y ago
    This is definitely true. In fact, this can be exploited to extract sensitive/private attributes about the training data from the learned models. This may become an issue for, e.g., AI in healthcare. "Overlearning Reveals Sensitive Attributes": https://arxiv.org/abs/1905.11742 https://arxiv.org/abs/1905.11742