4 ms·

A good popular take, but they, either intentionally or out of ignorance, omit newer, proven techniques like differential privacy.

buttercraft 3y ago
Or they don't want you collecting data in the first place, in which case the techniques are irrelevant.
- JohnFen 3y ago
  Entirely this. That data is so widely collected is the root problem.
pixl97 3y ago
In particular areas things like differential privacy may work, for example medical sets where there are lots of regulations and potential fines for the companies involved. But do you think your average ad tech company gives a fuck? They are going to keep the original data because that's where the money is. Yea, maybe they'll have privacy datasets they sell/release to other groups, but all the real data will remain in a database, and with most companies in this industry, be given to government agencies on demand.
- willseth 3y ago
  It's not because of the money. Most could make money just fine with properly anonymized data. They just don't care and/or don't want to take the effort or spend money to do it because the consequences for leaking private info are so minimal.
Forbo 3y ago
Still sounds potentially problematic. Per wikipedia: "Differential privacy provides a quantified measure of privacy loss and an upper bound and allows curators to choose the explicit trade-off between privacy and accuracy. It is robust to still unknown privacy attacks. However, it encourages greater data sharing, which if done poorly, increases privacy risk. Differential privacy implies that privacy is protected, but this depends very much on the privacy loss parameter chosen and may instead lead to a false sense of security. Finally, though it is robust against unforeseen future privacy attacks, a countermeasure may be devised that we cannot predict." If I am dependent on the curator to determine the level of privacy then I've already lost.
- JohnFen 3y ago
  I agree. DP is better than no privacy at all, but it certainly doesn't make me feel comfortable.
- 95014_refugee 3y ago
  You are already dependent on a set of curators to not simply outright lie and export the data as captured. This take lacks subtlety; unless you are going to abandon the set of functionality ("where is the best fried chicken near me?") that this sort of metadata facilitates, you need to make decisions about which curators you trust, and then participate in driving them to honesty and accountability. From my perspective this article leans too heavily into the FUD, and really doesn't succeed at keeping the call to action ticking over. On a good day, the EFF can be really good. Today, not so much so.
  - c0pium 3y ago
    The problem with the EFF is that it’s full of people who made up their minds about what’s ok and what isn’t over a decade ago, and they are largely just playing the hits now. No new material, and no consideration that things might change.
- c0pium 3y ago
  > …it encourages greater data sharing, which if done poorly, increases privacy risk. This is a really useful argument, because it’s the equivalent of the FDA’s “generally believed to be safe”. If you look into something and this is the risk you find, then it’s safe.
  - genewitch 3y ago
    GRAS. Btw.
godelski 3y ago
There's also Zero Knowledge Proofs which instead has a privacy guarantee Edit: Not trying to start a DP vs ZKP (or Homomorphic Encryption) flame war. We're all on the same side of privacy, right? Different tools for different situations.
jandrewrogers 3y ago
Techniques like differential privacy do not work for some types of data models, including many of the more interesting/risky ones. I've never seen a technique that can deliver an analytical model at scale that is both analytically effective and anonymous while also robust against sophisticated de-anonymization attacks. There is no good theoretical foundation to suggest that such things are possible. Most modern techniques for ensuring anonymization make assumptions that won't constrain sophisticated blackhats. They are good policy in a legal ass-covering sense and increase the cost required to de-anonymize but that is about it.
- willseth 3y ago
  Can you give an example? Global DP systems generally offer excellent privacy protection for the analytical distortion cost. Global DP is often not implemented simply due to workflow inconveniences but otherwise it’s great for analytical use cases.
- dfcowell 3y ago
  If your scale is large enough and you don’t care about identifying individuals, synthetic data does this fairly effectively.
orbital-decay 3y ago
Differential privacy does not anonymize your data. It's a (mathematically solid) instrument to make it coarse enough to only give up the part you want. The EFF is completely correct in their assessment - there's no way to make your data anonymous and usable by the third party at the same time. The root issue lies elsewhere. There's FHE, which does allow your data to be processed without accessing it, but it has prohibitive compute requirements and isn't practical for most purposes. I'm not sure if it allows the data to be usable enough without revealing it, either.
- Godel_unicode 3y ago
  > there's no way to make your data anonymous and usable by the third party at the same time. The This is absurd hyperbole, here’s a trivial proof. The US population is 51.1% female. If you live in the US your gender information is included in this data point, but it cannot be de-anonymized. It is however useful for determining TAM for many products. Anonymity is a spectrum, and there are many points along that spectrum which are solutions to usable data which also respects privacy.
  - JohnFen 3y ago
    Privacy is a spectrum, anonymity is not.

buttercraft 3y ago

Or they don't want you collecting data in the first place, in which case the techniques are irrelevant.

JohnFen 3y ago
Entirely this. That data is so widely collected is the root problem.

pixl97 3y ago

In particular areas things like differential privacy may work, for example medical sets where there are lots of regulations and potential fines for the companies involved. But do you think your average ad tech company gives a fuck? They are going to keep the original data because that's where the money is. Yea, maybe they'll have privacy datasets they sell/release to other groups, but all the real data will remain in a database, and with most companies in this industry, be given to government agencies on demand.

willseth 3y ago
It's not because of the money. Most could make money just fine with properly anonymized data. They just don't care and/or don't want to take the effort or spend money to do it because the consequences for leaking private info are so minimal.

Forbo 3y ago

Still sounds potentially problematic. Per wikipedia: "Differential privacy provides a quantified measure of privacy loss and an upper bound and allows curators to choose the explicit trade-off between privacy and accuracy. It is robust to still unknown privacy attacks. However, it encourages greater data sharing, which if done poorly, increases privacy risk. Differential privacy implies that privacy is protected, but this depends very much on the privacy loss parameter chosen and may instead lead to a false sense of security. Finally, though it is robust against unforeseen future privacy attacks, a countermeasure may be devised that we cannot predict." If I am dependent on the curator to determine the level of privacy then I've already lost.

JohnFen 3y ago
I agree. DP is better than no privacy at all, but it certainly doesn't make me feel comfortable.
95014_refugee 3y ago
You are already dependent on a set of curators to not simply outright lie and export the data as captured. This take lacks subtlety; unless you are going to abandon the set of functionality ("where is the best fried chicken near me?") that this sort of metadata facilitates, you need to make decisions about which curators you trust, and then participate in driving them to honesty and accountability. From my perspective this article leans too heavily into the FUD, and really doesn't succeed at keeping the call to action ticking over. On a good day, the EFF can be really good. Today, not so much so.
- c0pium 3y ago
  The problem with the EFF is that it’s full of people who made up their minds about what’s ok and what isn’t over a decade ago, and they are largely just playing the hits now. No new material, and no consideration that things might change.
c0pium 3y ago
> …it encourages greater data sharing, which if done poorly, increases privacy risk. This is a really useful argument, because it’s the equivalent of the FDA’s “generally believed to be safe”. If you look into something and this is the risk you find, then it’s safe.
- genewitch 3y ago
  GRAS. Btw.

godelski 3y ago

There's also Zero Knowledge Proofs which instead has a privacy guarantee Edit: Not trying to start a DP vs ZKP (or Homomorphic Encryption) flame war. We're all on the same side of privacy, right? Different tools for different situations.

jandrewrogers 3y ago

Techniques like differential privacy do not work for some types of data models, including many of the more interesting/risky ones. I've never seen a technique that can deliver an analytical model at scale that is both analytically effective and anonymous while also robust against sophisticated de-anonymization attacks. There is no good theoretical foundation to suggest that such things are possible. Most modern techniques for ensuring anonymization make assumptions that won't constrain sophisticated blackhats. They are good policy in a legal ass-covering sense and increase the cost required to de-anonymize but that is about it.

willseth 3y ago
Can you give an example? Global DP systems generally offer excellent privacy protection for the analytical distortion cost. Global DP is often not implemented simply due to workflow inconveniences but otherwise it’s great for analytical use cases.
dfcowell 3y ago
If your scale is large enough and you don’t care about identifying individuals, synthetic data does this fairly effectively.

orbital-decay 3y ago

Differential privacy does not anonymize your data. It's a (mathematically solid) instrument to make it coarse enough to only give up the part you want. The EFF is completely correct in their assessment - there's no way to make your data anonymous and usable by the third party at the same time. The root issue lies elsewhere. There's FHE, which does allow your data to be processed without accessing it, but it has prohibitive compute requirements and isn't practical for most purposes. I'm not sure if it allows the data to be usable enough without revealing it, either.

Godel_unicode 3y ago
> there's no way to make your data anonymous and usable by the third party at the same time. The This is absurd hyperbole, here’s a trivial proof. The US population is 51.1% female. If you live in the US your gender information is included in this data point, but it cannot be de-anonymized. It is however useful for determining TAM for many products. Anonymity is a spectrum, and there are many points along that spectrum which are solutions to usable data which also respects privacy.
- JohnFen 3y ago
  Privacy is a spectrum, anonymity is not.