4 ms·
"The training process behaves as a kind of one-way function. It is not possible to run a trained model backwards to reconstruct the input data; nor is it possib
by brilee 7y ago
"The training process behaves as a kind of one-way function. It is not possible to run a trained model backwards to reconstruct the input data; nor is it possible to “untrain” a model so that it will forget a specific part of its input."
I don't have a concrete reference, but my understanding is that it is quite likely to be possible to reconstruct outliers in the training data by inspecting the model's weights.
- bo1024 7y agoThis is a good point. We have some concerns and evidence that neural networks do memorize their training data. Also, the "untraining" idea is an open research question (I just saw a talk about it). We don't know exactly how to do it yet, definitely not in general, but "impossible" is too strong.