3 ms·
Is anyone aware of any work for how to apply differential privacy to language models? So the main question I have is let's say I'm working with sensitive data
by formalsystem 6y ago
Is anyone aware of any work for how to apply differential privacy to language models?
So the main question I have is let's say I'm working with sensitive data like emails or doctors notes. How can I train an ML model that would still learn something useful without leaking private data.
When I say "leak", an example would be I train an RNN on some company data email data and when I feed the RNN "$AMZN" the network would say SELL.
How can I quantify how much the model has learnt and how much privacy has been leaked.
- janhenr 6y agoCheck out Han Song's (MIT, most well-known for NN compression) Lab new paper on gradient leaking and follow-up work. https://arxiv.org/abs/1906.08935 https://arxiv.org/abs/1906.08935 I attended a talk by him recently and was very impressed by the work of his lab in this area.
- unixhero 6y agoQuantifying shareprice movements as a function of data leaks? That's brilliant!