3 ms·

You can put as many words as you want in quotes, but if you can point out a specific flaw in the secure aggregation paper, either in the algorithm description,

You can put as many words as you want in quotes, but if you can point out a specific flaw in the secure aggregation paper, either in the algorithm description, or the math, it would be taken a lot more seriously. It's like your responding to the Bitcoin paper by saying "proof of work" doesn't mean a thing because "hashing" isn't "money". There are papers published, critique those specifically, instead of relying on a handwavy respond to a comic. https://ai.google/research/pubs/pub47246 https://ai.google/research/pubs/pub47246

krick 7y ago
It is somehow understandable for a Google employee to take such a defensive position, but you either didn't read or didn't try to understand what I'm saying. I don't believe there is anything wrong with federated learning or that there are any problems with "secure aggregation" you are so fondly referring to. It's just that they don't mean shit, because, as I said several times already, you don't need to know that some data comes from my device, to learn something sensitive about me. It depends entirely on the application, no crypto has to be broken in order for that to happen.
- cromwellian 7y ago
  Well, I've been working on crypto and privacy since before the Web, 25+ years before I joined Google, released some of the first anonymizing proxy servers on the internet, designed one of the first mix-net PGP remailer networks, etc. I wouldn't be critiquing your response if you had something a little less handwavey to say. When people point out flaws in protocols, I like to see specific exploit examples, like the kind you'd put into a Spectre/Meltdown Advisory. If you said "out of order speculative execution in CPUs might eventually allow exploits", I may even have vaguely agreed, but without a concrete criticism, it's more of an "uneasy feeling" you have. I can make loose arguments too. Everything you do in this word leaks entropy. Your information is entangled with other people, leaving a wake behind you from the moment you're born. There's a gazillion side channels hanging off of you. So learning information about you is pretty much a given. The question is, is it relevant or important information? There's a huge difference between "learning something", "learning something sensitive about me (that I share with a large number of other people)" and "learning something sensitive about me that's individually traceable to me" Most people won't care if a Federated Learning model, using data from your phone, learns that people who stay up late, and search for Coke, also end up with diabetes, anymore than a double blind study learns about the risks of smoking and lung cancer -- the doctors have learned information about you (you're a smoker, and you have/don't have lung cancer), but they haven't learned that you, krick, are a smoker. The whole point of differential privacy, federated learning, and other techniques, is that aggregate statistics, and aggregate models can be learned without any personally traceable information. Now, you could argue that somewhere, deep within the logical depth of the weights of a DNN is some kind of personal information that could be deanonymized, but this is like a claim that you found a weakness in a hash function -- until you show it, it's just a claim, and mathematics and security research is full of wrong claims on both sides.

krick 7y ago

It is somehow understandable for a Google employee to take such a defensive position, but you either didn't read or didn't try to understand what I'm saying. I don't believe there is anything wrong with federated learning or that there are any problems with "secure aggregation" you are so fondly referring to. It's just that they don't mean shit, because, as I said several times already, you don't need to know that some data comes from my device, to learn something sensitive about me. It depends entirely on the application, no crypto has to be broken in order for that to happen.

cromwellian 7y ago
Well, I've been working on crypto and privacy since before the Web, 25+ years before I joined Google, released some of the first anonymizing proxy servers on the internet, designed one of the first mix-net PGP remailer networks, etc. I wouldn't be critiquing your response if you had something a little less handwavey to say. When people point out flaws in protocols, I like to see specific exploit examples, like the kind you'd put into a Spectre/Meltdown Advisory. If you said "out of order speculative execution in CPUs might eventually allow exploits", I may even have vaguely agreed, but without a concrete criticism, it's more of an "uneasy feeling" you have. I can make loose arguments too. Everything you do in this word leaks entropy. Your information is entangled with other people, leaving a wake behind you from the moment you're born. There's a gazillion side channels hanging off of you. So learning information about you is pretty much a given. The question is, is it relevant or important information? There's a huge difference between "learning something", "learning something sensitive about me (that I share with a large number of other people)" and "learning something sensitive about me that's individually traceable to me" Most people won't care if a Federated Learning model, using data from your phone, learns that people who stay up late, and search for Coke, also end up with diabetes, anymore than a double blind study learns about the risks of smoking and lung cancer -- the doctors have learned information about you (you're a smoker, and you have/don't have lung cancer), but they haven't learned that you, krick, are a smoker. The whole point of differential privacy, federated learning, and other techniques, is that aggregate statistics, and aggregate models can be learned without any personally traceable information. Now, you could argue that somewhere, deep within the logical depth of the weights of a DNN is some kind of personal information that could be deanonymized, but this is like a claim that you found a weakness in a hash function -- until you show it, it's just a claim, and mathematics and security research is full of wrong claims on both sides.