3 ms·
While I think you can definitely draw some parallels, differential privacy seems more targeted at metric collection. You have to be able to mutate the data in a
by jd20 10y ago
While I think you can definitely draw some parallels, differential privacy seems more targeted at metric collection. You have to be able to mutate the data in a way that it becomes non-identifying, without corrupting the answer in aggregate. Apple would still do all their training in the cloud.
In contrast, what Google's proposing is more like distributed training. In regular SGD, you'd iterate over a bunch of tiny batches, sequentially through your whole training set. Sounds like Google's saying each device becomes it's own mini-batch, and it beams up the result, and Google will average them all out in a smart way (I didn't read the paper, but this was the gist I got from the article).
Both ideas are in the same spirit, just the implementations are very different.
- Eridrus 10y agoDifferential Privacy is much more than what Apple's PR department says, differentially private SGD is already a thing.
- jd20 10y agoWell, forget Apple for a moment (that was just an example, since parent asked about them specifically): my point was what Google's describing is separate from differential privacy. There's no controlled noise or randomness being applied. They even say at the end of the paper: "While federated learning offers many practical privacy benefits, providing stronger guarantees via differential privacy, secure multi-party computation, or their combination is an interesting direction for future work." So, the "practical privacy benefits" here is referring to the dimensionality reduction from running the raw data thru the LSTM.