3 ms·

I'm not understanding this as homomorphic privacy. They take pains to say: > your device downloads the current model, improves it by learning from data on you

I'm not understanding this as homomorphic privacy. They take pains to say: > your device downloads the current model, improves it by learning from data on your phone, and then summarizes the changes as a small focused update. Only this update to the model is sent to the cloud, using encrypted communication, where it is immediately averaged with other user updates to improve the shared model. All the training data remains on your device, and no individual updates are stored in the cloud. (my emphasis on the word stored) Now there are lots of scholarly articles on reverse-engineering and rule-extraction from neural nets. So Google, having the diff can actually get some idea what it is you are trying to teach the net. They just promise not to.

azinman2 10y ago
And having worked at Google, I trust them on that.
fauigerzigerk 10y ago
>They just promise not to. Google makes the OS and the keyboard. If they wanted to run a keylogger on every device against the express wish of users they could. So I think the more important question is if someone else could steal or "legally" request that data from Google and recover my keystrokes.
memmcgee 10y ago
They say they have a technique called Secure Aggregation as well so that it can only decrypt the update if many users have sent their updates. I don't know how possible that would be to reverse engineer. Plus this would seem a computationally expensive way of conducting mass surveillance
pepve 10y ago
In the secure aggregation paper (linked in the post: http://eprint.iacr.org/2017/281 http://eprint.iacr.org/2017/281 ) they indeed hint at the possibility of extracting information from the diffs. So the protocol they propose cryptographically ensures that the individual diffs can not be learned. They do this by having each pair of clients generate a secret the size of the diff, and have one add it to their diff and the other subtract it. The clients then send that result to the server where they are summed, which cancels out all the secrets. The bulk of the protocol then deals with setting up the secrets and dealing with client drop outs, which appears to be the real challenge. Well, that's my fallible summary anyway, go read the paper. :-)

azinman2 10y ago

And having worked at Google, I trust them on that.

fauigerzigerk 10y ago

>They just promise not to. Google makes the OS and the keyboard. If they wanted to run a keylogger on every device against the express wish of users they could. So I think the more important question is if someone else could steal or "legally" request that data from Google and recover my keystrokes.

memmcgee 10y ago

They say they have a technique called Secure Aggregation as well so that it can only decrypt the update if many users have sent their updates. I don't know how possible that would be to reverse engineer. Plus this would seem a computationally expensive way of conducting mass surveillance

pepve 10y ago

In the secure aggregation paper (linked in the post: http://eprint.iacr.org/2017/281 http://eprint.iacr.org/2017/281 ) they indeed hint at the possibility of extracting information from the diffs. So the protocol they propose cryptographically ensures that the individual diffs can not be learned. They do this by having each pair of clients generate a secret the size of the diff, and have one add it to their diff and the other subtract it. The clients then send that result to the server where they are summed, which cancels out all the secrets. The bulk of the protocol then deals with setting up the secrets and dealing with client drop outs, which appears to be the real challenge. Well, that's my fallible summary anyway, go read the paper. :-)