3 ms·

Such obscuring is vulnerable against sidechannel attacks to re-link the record fragments. For example, misspellings, incidental geographic information, topics

Such obscuring is vulnerable against sidechannel attacks to re-link the record fragments. For example, misspellings, incidental geographic information, topics -- any pattern which is unusual, not anticipated by the modeler and deliberately obliterated. What links the AOL fiasco and this one is that both believe they have thought of everything important. They're wrong -- and there will always be a future attacker to prove it. You can't fight information theory. Differential Privacy is an excuse to get around sensible no-data policies -- by making irresponsible promises, it will result in more privacy violations, not less.

AdamSC1 9y ago
You're right these systems are currently flawed, and may remained flawed. But, the same is true for all systems. Don't get me wrong I agree in theory that every company should not retain any customer information - I work for one of a few companies that does that, but, I also know that many company's will not make that switch (example a bank, or a medical record) and in such cases that data should be as secure as possible. To counter the irresponsible promises you are talking about it'd be ideal to see compliance and security regulation like we see with HIPPA applied to all customer data, but until we evolve proper trust-less identification models and ways for users to self-secure and trustlessly validate their information then some businesses will always collect data.
frankmcsherry 9y ago
> You can't fight information theory. I totally agree with this statement, but I think you are confused about differential privacy. Its guarantees are information theoretic (specifically, a bound on the relative Bayes factors of any conclusion, with and without any individual record). You are obviously welcome to be skeptical, but much of what you've posted so far is not correct.
- rectang 9y ago
  Of course I don't dispute the math. I maintain that these guarantees will not be achieved in practice because they rely on impossibly airtight implementation and impossibly omniscient modeling.
  - frankmcsherry 9y ago
    Interesting. Do you have similar concerns about cryptography? Edit: to more strongly bind 'similar': would you also say of cryptography that it is "cynical academic malpractice"?
    - dsp1234 9y ago
      Not the person above, but I do, yes. Heartbleed, side channel attacks, etc. Implementation in the real world matters.
    - swordswinger12 9y ago
      There needs to be a word to describe the practice of arrogantly explaining a researcher's own results back to them on HN. Maybe HNsplaining?
    - rectang 9y ago
      I fully expect privacy disasters based on imperfect implementations of Differential Privacy. Do you not? Do the researchers not? The superior alternative is to avoid sharing sensitive datasets and avoid keeping data whenever possible. No such alternative exists for many applications of cryptography. But we live in an era where organizations find our private data impossibly tempting and are content to sacrifice the rights of individuals so long as they can't fight back. This research gives such entities the excuse to build tools that should not be built and publish data that should not be published. By saying "OK now it's safe (if you did everything right)" rather than "don't do that", it is the enabler of future privacy fiascos. The answer, if there is one, is probably legislative: hold entities criminally liable for data breach. Should such legislation pass, I wonder how much interest in this research will wane.
      - frankmcsherry 9y ago
        If you want to rip in to Apple or Google or Uber for claiming they should have a pass for using privacy tech, feel free. Understand that this is distinct from most research on differential privacy. The US Census collects demographic data about as much of the population as they can manage, and releases summary data in a large part to support enforcement of the Civil Rights Act. They have a privacy mandate, but also the obligation to provide information in support of the rights of subpopulations (e.g. Equal Protection). So what's your answer here? A large fraction of the population gets disenfranchised if you go with "avoid sharing the datasets". You end up with similar issues in preventative medicine, epidemiology, public health, where there is a real social benefit to analyzing data, and where withholding data has a cost that hasn't shown up yet in your analysis. Understanding the trade-off is important, and one can come to different conclusions when the subjects are civil rights versus cell phone statistics. But you are wrong to be upset that math allows the trade-off to exist.

AdamSC1 9y ago

You're right these systems are currently flawed, and may remained flawed. But, the same is true for all systems. Don't get me wrong I agree in theory that every company should not retain any customer information - I work for one of a few companies that does that, but, I also know that many company's will not make that switch (example a bank, or a medical record) and in such cases that data should be as secure as possible. To counter the irresponsible promises you are talking about it'd be ideal to see compliance and security regulation like we see with HIPPA applied to all customer data, but until we evolve proper trust-less identification models and ways for users to self-secure and trustlessly validate their information then some businesses will always collect data.

frankmcsherry 9y ago

> You can't fight information theory. I totally agree with this statement, but I think you are confused about differential privacy. Its guarantees are information theoretic (specifically, a bound on the relative Bayes factors of any conclusion, with and without any individual record). You are obviously welcome to be skeptical, but much of what you've posted so far is not correct.

rectang 9y ago
Of course I don't dispute the math. I maintain that these guarantees will not be achieved in practice because they rely on impossibly airtight implementation and impossibly omniscient modeling.
- frankmcsherry 9y ago
  Interesting. Do you have similar concerns about cryptography? Edit: to more strongly bind 'similar': would you also say of cryptography that it is "cynical academic malpractice"?
  - dsp1234 9y ago
    Not the person above, but I do, yes. Heartbleed, side channel attacks, etc. Implementation in the real world matters.
  - swordswinger12 9y ago
    There needs to be a word to describe the practice of arrogantly explaining a researcher's own results back to them on HN. Maybe HNsplaining?
  - rectang 9y ago
    I fully expect privacy disasters based on imperfect implementations of Differential Privacy. Do you not? Do the researchers not? The superior alternative is to avoid sharing sensitive datasets and avoid keeping data whenever possible. No such alternative exists for many applications of cryptography. But we live in an era where organizations find our private data impossibly tempting and are content to sacrifice the rights of individuals so long as they can't fight back. This research gives such entities the excuse to build tools that should not be built and publish data that should not be published. By saying "OK now it's safe (if you did everything right)" rather than "don't do that", it is the enabler of future privacy fiascos. The answer, if there is one, is probably legislative: hold entities criminally liable for data breach. Should such legislation pass, I wonder how much interest in this research will wane.
    - frankmcsherry 9y ago
      If you want to rip in to Apple or Google or Uber for claiming they should have a pass for using privacy tech, feel free. Understand that this is distinct from most research on differential privacy. The US Census collects demographic data about as much of the population as they can manage, and releases summary data in a large part to support enforcement of the Civil Rights Act. They have a privacy mandate, but also the obligation to provide information in support of the rights of subpopulations (e.g. Equal Protection). So what's your answer here? A large fraction of the population gets disenfranchised if you go with "avoid sharing the datasets". You end up with similar issues in preventative medicine, epidemiology, public health, where there is a real social benefit to analyzing data, and where withholding data has a cost that hasn't shown up yet in your analysis. Understanding the trade-off is important, and one can come to different conclusions when the subjects are civil rights versus cell phone statistics. But you are wrong to be upset that math allows the trade-off to exist.