6 ms·

All right, I'm cynical as all heck about ad companies and privacy, but this has me optimistic. Somebody disillusion me, why shouldn't I be optimistic?

deleted 7y ago
[deleted]
carlosdp 7y ago
It could always end up being just fluff, but given it's an active research area and they've open sourced a framework based on the concept already, plus the fact the model is cost-advantageous given you get to offload training to a fleet of customer devices instead of paying for your own servers, it could be the real deal. I started reading it looking at it from a cynical view, but ended it with a "hmm... this could actually work," especially after reading about Secure Aggregation. That's badass.
patrickthebold 7y ago
Looks like everyone gets the same model. So it won't be used for things like targeted ads. Cynical view is that it's only used when Google doesn't want your individual data. This will help muddy the waters in discussions about privacy.
- carlosdp 7y ago
  Sure, it doesn't address that problem head on, however if they moved most stuff to a federated model, they're only a hop, skip, and a jump away from doing ad-selection on-device too, right? I think that if the model proves successful, it could end up proving out the concept enough to incentivize at least trying it out.
  - singron 7y ago
    There are other reasons not to do things on-device. For instance, being able to pick your own ads might hinder the ability to defend against ad-viewing bots. Also, traditional ad-tech also has to deal with meeting global target spend rates (e.g. if client A pays for 1000 impressions, and client B pays for 9000 impressions, then 90% of the ads must be for client B even if every device would prefer ads for client A). Usually there are realtime databases keeping track of impression counts for all the campaigns and it would be infeasible to synchronize that with every device so that they could make local targeting decisions. I.e. you can't necessarily make the decision locally if it's not local problem, and ad-targeting usually isn't a local problem.
- jameslevy 7y ago
  Just because everyone gets the same model doesn't mean you can't have targeted ads. It just means the prediction model isn't modeled specific to your behavior. It can still make personalized predictions based on your search input, email content, etc.
cavisne 7y ago
Well the cynical view would be. 1) This still lets you have personalized models, just trained on more than 1 user, thats fine at google's scale anyway 2) Their competitors (FB, AMZN) dont have the edge compute (Android) to do this, and to a lesser degree don't have the ML stack (however Android implements this at the API level will be very Tensorflow focused) 3) Now google can push for privacy regulations that prevent FB and AMZN from storing your raw data 4) Profit That said theres nothing stopping FB doing federated learning within their app on mobile, I just don't think they have the privacy background to bother.
- gbrown 7y ago
  If #3 happens I'd be shocked (and pleased).
  - pm90 7y ago
    They would be absolutely compelled to by market forces. The entire ad-tech industry would be wiped out if something like this becomes more common. You would need an army of highly trained machine learning engineers to build complex systems that would work great but still be efficient. Guess who is the only company which has that army? The cynic in me sees this as a great play by the Googz to cut the Amazon-adtech venture in the bud, and to establish and maintain dominance over the adtech business, and advertising in general.
- defen 7y ago
  > Their competitors (FB, AMZN) dont have the edge compute (Android) to do this It's not Android scale but Amazon has sold 100 million Alexas.
  - hatsunearu 7y ago
    Doubt Alexas have enough juice to do on device training...
nixpulvis 7y ago
Well the skeptic in me doesn't want my model touched by other peoples models. Of course it depends on the application... For example, I might not want my keyboard autocomplete learning from others, but I might want my self driving car to do it. For applications where we want a common model, I see no other way to do it but this. The idea companies collect massive stockpiles of data forever is infeasible.
krick 7y ago
I think this is pretty cool from the technical perspective, but indeed I don't see the reason to be optimistic from the "privacy concerns" perspective. In fact, if this is seriously gonna be used as a "better privacy" argument (as some people seem to be already doing in this very thread), I'm calling it a PR victory for the "bad guys". First off, if you were worried about what Android was sending to Google, there's no reason to believe it's going to stop. In fact, I believe that the biggest problem always was the (carefully cultivated) confusion about what data is being sent: you have a hundred of menus to "opt-out" of something and it's not even exactly clear if it changes anything. In fact, we know for a fact, that when "opting out" in some cases more data is being sent. And even if we are not entirely happy about it, most of us still allow this to happen, because there's nothing we can directly do to prevent it and everybody says "well, I do need a smartphone after all, right?" (yeah-yeah, somebody doesn't, but we are not talking about the weird minority here) And all of it happens when it's relatively straightforward to see what data is being sent, because of minimal aggregation on the device. And still even somewhat technically-minded people don't really know what Google (Facebook, Amazon, whatever) really knows about them. Second, what really is "federated learning"? Well, let's imagine no humans speak Chinese, but there is this program (owned by Google), that does. And speaking Chinese is how it actually operates internally, when deciding to show this or that ad to you, or sending a ballistic missile to your location. So, in order for it to learn, we normally were sending English sentences, which were translated to Chinese server-side. Federated learning is when they are translated to Chinese client-side (which might be considered "lossy" conversion, but to what degree is not really specified), and then sent to Google to be aggregated. So, yeah, no raw data has been sent, but the central Chinese-speaking machine still somehow knows it all. What exactly it knows, depends on what we really meant by "translating into Chinese" in our metaphor. But effectively, we just offloaded some processor work to the client side, which, as I said, seems really cool to me from the technical perspective, but there's no way it automatically protects us from anything. Third is basically 1 + 2: we didn't know what is being sent when it was all raw-data, we will know even less, when it's client-side aggregated in some unintelligible-for-the-humans way. And it scares me even more, because if it allows some PR victories for the Google&Friends, then sky is the limit for what more surveillance can be done this way. I mean, if it would be publicly known that Android sends all the sound and all the image from your mic & camera to Google, I think (I hope!) that people would seriously oppose to that. But if it's not the real images, but just some matrix of weights, learnt from them — it might be less clear if anybody has to object to that. And I think they absolutely have to! Because if we don't make any very restrictive assumptions about what we mean by "learning" in this very specific case, then the only thing that matters is that the "central brain" still saw all these images, it just isn't known what exactly it "remembered". After all, we, humans, also don't store all the pictures we've seen in our brains: it doesn't make you much happier if I saw your transaction history (or whatever else you don't want me to know), because it was never the picture I was after, but only the "aggregated info".
frenchtoastto 7y ago
Well if large companies have access to metadata like this aggregate or not they know what people are doing, looking at, and talking about. Which allows them to play the financial markets like a drum. Why do they need the aggregate data? To make money plane and simple, tis capitalism 101.
arnioxux 7y ago
I wonder how they are going to handle security/abuse? Once the training is on end user devices, can't the user give fake results? For example back when recaptcha was introduced, trolls tried to transcribe everything as the n-word (since you only had to get one of the two words correct). These cases can obviously be noticed and fixed but it's harder now that the training data is opaque right?

deleted 7y ago

[deleted]

carlosdp 7y ago

It could always end up being just fluff, but given it's an active research area and they've open sourced a framework based on the concept already, plus the fact the model is cost-advantageous given you get to offload training to a fleet of customer devices instead of paying for your own servers, it could be the real deal. I started reading it looking at it from a cynical view, but ended it with a "hmm... this could actually work," especially after reading about Secure Aggregation. That's badass.

patrickthebold 7y ago

Looks like everyone gets the same model. So it won't be used for things like targeted ads. Cynical view is that it's only used when Google doesn't want your individual data. This will help muddy the waters in discussions about privacy.

carlosdp 7y ago
Sure, it doesn't address that problem head on, however if they moved most stuff to a federated model, they're only a hop, skip, and a jump away from doing ad-selection on-device too, right? I think that if the model proves successful, it could end up proving out the concept enough to incentivize at least trying it out.
- singron 7y ago
  There are other reasons not to do things on-device. For instance, being able to pick your own ads might hinder the ability to defend against ad-viewing bots. Also, traditional ad-tech also has to deal with meeting global target spend rates (e.g. if client A pays for 1000 impressions, and client B pays for 9000 impressions, then 90% of the ads must be for client B even if every device would prefer ads for client A). Usually there are realtime databases keeping track of impression counts for all the campaigns and it would be infeasible to synchronize that with every device so that they could make local targeting decisions. I.e. you can't necessarily make the decision locally if it's not local problem, and ad-targeting usually isn't a local problem.
jameslevy 7y ago
Just because everyone gets the same model doesn't mean you can't have targeted ads. It just means the prediction model isn't modeled specific to your behavior. It can still make personalized predictions based on your search input, email content, etc.

cavisne 7y ago

Well the cynical view would be. 1) This still lets you have personalized models, just trained on more than 1 user, thats fine at google's scale anyway 2) Their competitors (FB, AMZN) dont have the edge compute (Android) to do this, and to a lesser degree don't have the ML stack (however Android implements this at the API level will be very Tensorflow focused) 3) Now google can push for privacy regulations that prevent FB and AMZN from storing your raw data 4) Profit That said theres nothing stopping FB doing federated learning within their app on mobile, I just don't think they have the privacy background to bother.

gbrown 7y ago
If #3 happens I'd be shocked (and pleased).
- pm90 7y ago
  They would be absolutely compelled to by market forces. The entire ad-tech industry would be wiped out if something like this becomes more common. You would need an army of highly trained machine learning engineers to build complex systems that would work great but still be efficient. Guess who is the only company which has that army? The cynic in me sees this as a great play by the Googz to cut the Amazon-adtech venture in the bud, and to establish and maintain dominance over the adtech business, and advertising in general.
defen 7y ago
> Their competitors (FB, AMZN) dont have the edge compute (Android) to do this It's not Android scale but Amazon has sold 100 million Alexas.
- hatsunearu 7y ago
  Doubt Alexas have enough juice to do on device training...

nixpulvis 7y ago

Well the skeptic in me doesn't want my model touched by other peoples models. Of course it depends on the application... For example, I might not want my keyboard autocomplete learning from others, but I might want my self driving car to do it. For applications where we want a common model, I see no other way to do it but this. The idea companies collect massive stockpiles of data forever is infeasible.

krick 7y ago

I think this is pretty cool from the technical perspective, but indeed I don't see the reason to be optimistic from the "privacy concerns" perspective. In fact, if this is seriously gonna be used as a "better privacy" argument (as some people seem to be already doing in this very thread), I'm calling it a PR victory for the "bad guys". First off, if you were worried about what Android was sending to Google, there's no reason to believe it's going to stop. In fact, I believe that the biggest problem always was the (carefully cultivated) confusion about what data is being sent: you have a hundred of menus to "opt-out" of something and it's not even exactly clear if it changes anything. In fact, we know for a fact, that when "opting out" in some cases more data is being sent. And even if we are not entirely happy about it, most of us still allow this to happen, because there's nothing we can directly do to prevent it and everybody says "well, I do need a smartphone after all, right?" (yeah-yeah, somebody doesn't, but we are not talking about the weird minority here) And all of it happens when it's relatively straightforward to see what data is being sent, because of minimal aggregation on the device. And still even somewhat technically-minded people don't really know what Google (Facebook, Amazon, whatever) really knows about them. Second, what really is "federated learning"? Well, let's imagine no humans speak Chinese, but there is this program (owned by Google), that does. And speaking Chinese is how it actually operates internally, when deciding to show this or that ad to you, or sending a ballistic missile to your location. So, in order for it to learn, we normally were sending English sentences, which were translated to Chinese server-side. Federated learning is when they are translated to Chinese client-side (which might be considered "lossy" conversion, but to what degree is not really specified), and then sent to Google to be aggregated. So, yeah, no raw data has been sent, but the central Chinese-speaking machine still somehow knows it all. What exactly it knows, depends on what we really meant by "translating into Chinese" in our metaphor. But effectively, we just offloaded some processor work to the client side, which, as I said, seems really cool to me from the technical perspective, but there's no way it automatically protects us from anything. Third is basically 1 + 2: we didn't know what is being sent when it was all raw-data, we will know even less, when it's client-side aggregated in some unintelligible-for-the-humans way. And it scares me even more, because if it allows some PR victories for the Google&Friends, then sky is the limit for what more surveillance can be done this way. I mean, if it would be publicly known that Android sends all the sound and all the image from your mic & camera to Google, I think (I hope!) that people would seriously oppose to that. But if it's not the real images, but just some matrix of weights, learnt from them — it might be less clear if anybody has to object to that. And I think they absolutely have to! Because if we don't make any very restrictive assumptions about what we mean by "learning" in this very specific case, then the only thing that matters is that the "central brain" still saw all these images, it just isn't known what exactly it "remembered". After all, we, humans, also don't store all the pictures we've seen in our brains: it doesn't make you much happier if I saw your transaction history (or whatever else you don't want me to know), because it was never the picture I was after, but only the "aggregated info".

frenchtoastto 7y ago

Well if large companies have access to metadata like this aggregate or not they know what people are doing, looking at, and talking about. Which allows them to play the financial markets like a drum. Why do they need the aggregate data? To make money plane and simple, tis capitalism 101.

arnioxux 7y ago

I wonder how they are going to handle security/abuse? Once the training is on end user devices, can't the user give fake results? For example back when recaptcha was introduced, trolls tried to transcribe everything as the n-word (since you only had to get one of the two words correct). These cases can obviously be noticed and fixed but it's harder now that the training data is opaque right?