4 ms·
Well you obviously can't fully reverse engineer, since whatever model update being sent is far far smaller than the overall data. Now, could you theoretically e
by ehsankia 10y ago
Well you obviously can't fully reverse engineer, since whatever model update being sent is far far smaller than the overall data. Now, could you theoretically extract "some" data? Maybe, but it still is strictly better than sending all of the data.
- sdenton4 10y agoThe 'sentiment neuron' two posts over gives an indication of what this could look like. 'Oh, I see you're giving a strong positive update to the porn neuron...' More generally, there's the notion that a sufficiently complex model encodes the training data. There's been work on extracting training data (or highly deformed versions of it) from a fully trained neural network. We should be under no illusions that networks offer cryptographically strong protections to their memories. It's simply not a design goal.
- justonepost 10y agoYep. Basically you just reverse the network and predict the source - if not done right.
- argonaut 10y agoNo, generally models are higher dimensional than their inputs (AKA deep learning models generally have more parameters than the number of features).