3 ms·

Let's say i'm collecting a simple yes / no piece of data such as... did this user open Google Chrome today. Every day my analytics engine sends the data back up

Let's say i'm collecting a simple yes / no piece of data such as... did this user open Google Chrome today. Every day my analytics engine sends the data back up: yes, no, no, yes, yes, yes, no. Someone could look at this data and know whether you used Chrome or not on a given day, but when Apple sends the data up it randomly flips the answer for a certain number of data points in such a way that the flipping effects will cancel each other out at scale and the overall stats will be more or less the same. Now, even if I were able to look at all the data your computer sent up about your Chrome usage I can't actually be sure which days you used Chrome because I know some of the answers are flipped. The more changes get made, the more privacy the user has.

mirimir 9y ago
OK, thanks :) But still, there are some questions that you'd arguably never want to say "yes" to. Such as, did you visit some verboten site (terrorist, child porn, etc) today? So how can an algorithm "know" which questions it's safe to use differential privacy with, and which it isn't? Or would you argue that it's safe enough to use differential privacy with even such questions?
- orf 9y ago
  I don't think this approach applies in such situations, and wouldn't be a good idea if it did. It may help with categories of sites, like "did you visit a news site today" or "did you watch porn".
  - mirimir 9y ago
    Systems that don't deal well with edge cases aren't so great. For people who don't really need to care about privacy, I guess that differential privacy is good enough. But there's a gotcha there, for people who ought to care, but are clueless. I'm reminded of that ex cop in Philadelphia, who believed Freenet's claims about plausible deniability. Edit: A prudent standard for calling something "[foo] privacy" is arguably PGP.
- yorwba 9y ago
  You're not really saying "yes", you're saying "I either visited this site or my coin came up heads.". Of course this only works if the other person believes that you flipped a coin beforehand, but in this case it's part of the data-collection procedure, so not much trust is necessary.
- geofft 9y ago
  Presumably almost nobody visits the verboten sites, and so if you answer "yes", it's far more likely that you answered "yes" because your coin came up tails than because you actually visited it. Or, in other words, the question is not "Did you visit this site," the question is "Did your coin come up tails or did you visit this site," which is a perfectly safe question to say "yes" to.
frankmcsherry 9y ago
> The more changes get made, the more privacy the user has. This is a minor, pedantic point, but what you really mean to say is "the closer the changes get to 50%, the more privacy the user has". If you change all results, then it is easy to flip them all back. This distinction trips up several folks, where the research world initially believed (and some in official stats still believe[1]) that a part of privacy is literally not publishing the true answer (e.g. above: literally flip every output). What you actually want is Pr[output | input] ~= Pr[output | input'] which may mean that you should leave things alone, which feels weird but is important. [1]: Noise addition is a common way to obscure real-valued data, and some official stats bureaus have the ridiculous rule that "you always add noise, and you never add less than X in absolute value", leading to releases where you can be 100% confident that the true value is not in a range around the published number.

mirimir 9y ago

OK, thanks :) But still, there are some questions that you'd arguably never want to say "yes" to. Such as, did you visit some verboten site (terrorist, child porn, etc) today? So how can an algorithm "know" which questions it's safe to use differential privacy with, and which it isn't? Or would you argue that it's safe enough to use differential privacy with even such questions?

orf 9y ago
I don't think this approach applies in such situations, and wouldn't be a good idea if it did. It may help with categories of sites, like "did you visit a news site today" or "did you watch porn".
- mirimir 9y ago
  Systems that don't deal well with edge cases aren't so great. For people who don't really need to care about privacy, I guess that differential privacy is good enough. But there's a gotcha there, for people who ought to care, but are clueless. I'm reminded of that ex cop in Philadelphia, who believed Freenet's claims about plausible deniability. Edit: A prudent standard for calling something "[foo] privacy" is arguably PGP.
yorwba 9y ago
You're not really saying "yes", you're saying "I either visited this site or my coin came up heads.". Of course this only works if the other person believes that you flipped a coin beforehand, but in this case it's part of the data-collection procedure, so not much trust is necessary.
geofft 9y ago
Presumably almost nobody visits the verboten sites, and so if you answer "yes", it's far more likely that you answered "yes" because your coin came up tails than because you actually visited it. Or, in other words, the question is not "Did you visit this site," the question is "Did your coin come up tails or did you visit this site," which is a perfectly safe question to say "yes" to.

frankmcsherry 9y ago

> The more changes get made, the more privacy the user has. This is a minor, pedantic point, but what you really mean to say is "the closer the changes get to 50%, the more privacy the user has". If you change all results, then it is easy to flip them all back. This distinction trips up several folks, where the research world initially believed (and some in official stats still believe[1]) that a part of privacy is literally not publishing the true answer (e.g. above: literally flip every output). What you actually want is Pr[output | input] ~= Pr[output | input'] which may mean that you should leave things alone, which feels weird but is important. [1]: Noise addition is a common way to obscure real-valued data, and some official stats bureaus have the ridiculous rule that "you always add noise, and you never add less than X in absolute value", leading to releases where you can be 100% confident that the true value is not in a range around the published number.