18 ms·

Big data may not know your name, but it knows everything else

190 points by nemoniac 5y ago

dang 5y ago
We've merged https://news.ycombinator.com/item?id=29734713 https://news.ycombinator.com/item?id=29734713 into this thread since the previous submission wasn't to the original source. That's why some comment timestamps are older.
jmatthews 5y ago
Without digging in to details, I just want to agree that reversing into a clients PII, while not trivial, is certainly possible, and depending on the class of data, one can achieve fidelity levels in the high 80 percentiles.
lb1lf 5y ago
-Anecdotally, at a former employer we had an annual questionnaire used to estimate how content we were. The results, we were assured, would only be used in aggregate after having been anonymized. I laughed quite hard when the results were back - the 'Engineer, female, 40-49yrs, $SITE' in the office next door wasn't as amused. All her responses had been printed in the report. Sample size: 1.
- omegalulw 5y ago
  Isn't this a trivial example though? You require bucket size >= 10 say. Full on differential privacy is meant to address harder problems. For example, if you assume that there was a minor bug and people were asked to take the mandatory yearly survey again in a week. Now if you assume that the responses of people who responded before will stay the same when they re-respond, then by taking the difference between both results you can identify some specific people by looking at which bucket counts changed - this would be how the new guy who joined in that week voted. Differential privacy techniques are meant to anonymise data in way that you can't even do this.
  - musingsole 5y ago
    It's trivial to fix in retrospect, nearly impossible to anticipate all similar corner cases ahead of time, and -- most importantly -- indicative of the type of logical bugs our systems of the world have that permit hacking from script-kiddies to state actors.
    - saghm 5y ago
      It may be nearly impossible to anticipate _all_ corner cases, but come on, GP was talking about there only being a sample size of 1; this is super easy to notice and then just not include in the results.
- sqrt17 5y ago
  At our (fairly large) company, you can query by team (and maybe job role) but it will hide responses where the sample size is smaller than a set number (I think 8 or 10). So yes it can be done but people have to actually care about it. The cautionary tale about k-anonymity (from Aaron Schwartz's book I think) is when the behavior of aggregates is also something that should be kept privates - the example was that the morning run of an army base in a foreign country was revealed because enough people did this with their smartwatches on that it formed a neat cluster.
  - teraku 5y ago
    In Germany (I think all of the EU), a dataset can only be published if the sample size is at least >=7.
    - williamtrask 5y ago
      Just fwiw sample size isn’t a robust defence against this kind of attack. Check out Differential Privacy.
  - ocdtrekkie 5y ago
    More details on the Strava Run incident: https://www.bbc.com/news/technology-42853072 https://www.bbc.com/news/technology-42853072
  - pfraze 5y ago
    Isn’t location data particularly easy to de-anonymize? I remember reading some research that because people tend to be so consistent with their location, you could deanonymize most people in a dataset with 3 random location samples through the day
    - morganherlocker 5y ago
      This is likely the paper you are referring to: https://www.nature.com/articles/srep01376 https://www.nature.com/articles/srep01376 It states that 95% of people can be identified from just 4 location samples.
- AstralStorm 5y ago
  Even without sample size having these aggregated results makes it very easy to predict who picked what with a modicum of extra information. (Even silly binned personality type.)
- rotten 5y ago
  Never answer those honestly. More than half the time they aren't there to "help management understand how to do better", rather to purge people who aren't happy. "We don't need employees who don't love it here."
  - lb1lf 5y ago
    -Oh, we were already deemed beyond redemption - we'd been a small company, quite successful in our narrow niche, only to be bought up by $MEGACORP. That was a culture clash. Big time. Nevertheless, I thought the same thing you did and filled in my questionnaire so that my answers created a nice, symmetrical pattern - it looked almost like a pattern for an arts&crafts project...
  - vorpalhex 5y ago
    I always say I am unhappy and would take another offer in a heatbeat, and so far that strategy has worked well for me - I usually get offered a good salary bump amd bonus every year. Obviously YMMV.
- AlexTWithBeard 5y ago
  Can confirm. I used to have a team and at some point they all had to submit their feedback on my performance. The answers were then fed back to me unattributed, but it was pretty obvious who wrote what.
  - vanusa 5y ago
    And - were there "consequences" for these people?
    - AlexTWithBeard 5y ago
      No consequences. There was nothing I didn't know before. EDIT: I first wanted to say that in any way there would be no consequences, but "there will be no consequences no matter what people write about me" is probably too strong of a statement for anyone to make.
      - ipaddr 5y ago
        Were there positive comments you didn't expect?
        AlexTWithBeard 5y ago
        Kinda. I expected some people to leave a good feedback, but theirs was way beyond my expectations.
benreesman 5y ago
One click removed from an original source that is soaked in second-rate adtech crap. To my dying day I will regret being one of the architects of this insidious mechanism. The problem with trying to evade, or defeat, or even sidestep this stuff is that latent-rep embeddings break human intuition in their effectiveness. There was a time when the uniqueness of one’s signature could move money. Those pen twitches are still there to see in what order you click on links.
- jaytaylor 5y ago
  Having a conscience is good, but don't be too hard on yourself old friend.
- kurthr 5y ago
  Just wait until they have your unconscious eye-twitches. Foveal rendering is required for adequate resolution/refresh of VR/AR due to the bandwidths and GPU calcs involved (e.g. 6kx6kxRGBx2eyesx120Hz=~200Gb/s). Updating only the 20-30deg around the eye's focus allows reducing this by >10x which reduces power/weight and GPU cost dramatically.
aledalgrande 5y ago
To me it is crazy selling that data is even legal.
- alkonaut 5y ago
  To me it’s crazy that people go to work doing this. Even if you can negotiate the moral issues, how to people find things like adtech interesting? Some people seem legitimately excited when saying their product helps customers show “relevant experiences” and so on.
  - shadowgovt 5y ago
    Because human behavior is absolutely fascinating, as is the fact it can be shaped.
  - 1vuio0pswjnm7 5y ago
    If I am not mistaken, yesterday there was a top comment that started "Adtech veteran here." Perhaps surveilling people on the internet distorts one's sense of time, or, alternatively, some folks will just say anything to defend what they are doing.
  - analognoise 5y ago
    "It is difficult to get a man to understand something when his salary depends upon his not understanding it." Upton Sinclair
  - chupy 5y ago
    Working in adtech was interesting for me because I learned how to work with distributed systems, thought me the importance of optimizing, scaling, instrumentation etc. It's extremely satisfying shaving a ms or two in response times in an app when you have 15 ms in total to respond. I built for example a data pipeline that ingests Bluekai/Oracle data cloud data and from a big data perspective it's an amazing experience: from dealing with ingesting billions of records per day to making sure you can build audiences based on those records in near real time. From a privacy perspective it's very scary stuff. When you look at what data Bluekai has, there are hundreds of millions of profiles scattered around hundreds of thousands of segments. Basically you have segments from frozen food shoppers to people that bought a mercedes a class model 2020. P.S.I can say also the same thing about Google, FB etc. A lot of people that work there are basically working in adtech, especially in Google.
MrDresden 5y ago
Anecdotally, the only time I've seen a truly anonymized database was in a european genetics research company, due mainly to the rightly high amount of regulation required in the medical field. There was a whole separate legal entity, with its own board, that did the phenotype measurement gathering and stored the data in a big database on premise. The link between those measurements and the individual's personal identifiable record was then stored in a separate airgapped database which had cryptographic locks implemented (on the data and physical access to the server) so accessing the data took the physical presence of the privacy officers of each of the two companies (the measurement lab and the research lab) and finally what I found at the time to be the unique move; a representative from the state run privacy watchdog. To be able to backtrack the data to a person, there was always going to be a need to go through the watchdog. Required, not just legally mandated. All of the measurement data that was stored in the database came from very restricted input fields in the custom software that was made on prem (no long form text input fields for instance, where identifying data could be put in accidentally), and there was a lot of thought put into the design of the UI to limit the possibility that anyone could put identifiable data into the record. For instance numerical ranges for a specific phenotype where all prefilled in a dropdown, so as to keep user key input to a minimum. Much of the data also came from direct connections to the medical equipment (I wrote a serial connector for a Humphrey medical eye scanner that parsed the results straight into the software, skipping the human element all together). This didn't make for the nicest looking software (endless dropdowns or scales), but it fulfilled its goal of functionality and privacy concerns perfectly. The measurement data would then go through many automatic filters and further anonymizing processes before being delivered through a dedicated network pipeline (configured through the local isp to be unidirectional) to the research lab. Is this guaranteed to never leak any private information? No, nothing is 100%. This comes damn near close to it, but ofcourse would not work in most other normal business situations.
- Hendrikto 5y ago
  > To be able to backtrack the data to a person, there was always going to be a need to go through the watchdog. The assumption in the deanonymization literature is that this data is unavailable. So no, you don‘t need to go through any watchdog.
sanxiyn 5y ago
Remember that 33 bits of entropy are enough to identify everyone. It may not be legally so, but any data with 33 bits of entropy is technically PII, and you should treat it as such.
- kortex 5y ago
  > any data with 33 bits of entropy is technically PII, Well yes, but actually no. I can run UUIDgen 33 times and put it on pastebin, via tor. That does not mean I have 128 peoples' worth of PII. In fact I have ~0 bits of PII. If I were to paste one of those numbers in this comment, now they are all linked to kortex, and however much entropy that username has (call it 32, this account is leaky). But you are still short 127 "PIIs". The rest of the entropy means nothing. Conversely, a US phone number is <24 bits, but 100% PII. It's not 33 bits of "entropy". It's 33 bits worth of a distribution which can be correlated to an identity. If it's not correlated, it's not PII.
- konschubert 5y ago
  That makes no sense, sorry. Ok, 2^33 > world population, but that doesn't mean that the string "Hello world" is PII.
  - AstralStorm 5y ago
    Unless someone is actually called Hello World. Or perhaps Bobby Tables. ;)
  - halhen 5y ago
    That depends on the encoding, does it not? The binary sequence equal to ASCII "Hello world" might well be PII with many different encodings. By accident, of course, but nevertheless 33 bits of information would be enough.
  - stillicidious 5y ago
    33 bits of entropy, not just 33 bits
    - deleted 5y ago
agsnu 5y ago
If you're interested in this topic, I recommend the chapter on Inference Control in Ross Anderson's excellent book "Security Engineering". It's one of the ones which is freely available on his web site https://www.cl.cam.ac.uk/~rja14/Papers/SEv3-ch11-7sep.pdf https://www.cl.cam.ac.uk/~rja14/Papers/SEv3-ch11-7sep.pdf
lrem 5y ago
In Google we have a bunch of researchers on anonymity and that whole thing is _hard_. I vaguely remember supporting, a couple years ago, a pipeline where logs stripped out of "all PII" came at one end, aggregated data out of the middle... Into an anonymity verifier, that then redirected much of it into /dev/null, because a technique is known to de-anonymise it. And the research on differential anonymity advanced quite a bit in the meantime.
- geoduck14 5y ago
  Did that stuff work?
float4 5y ago
Two things I'd like to say here 1. All anonymisation algorithms (k-anonymity, l-divergence, t-closeness, e-differential privacy, (e,d)-differential privacy, etc.) have, as you can see, at least one parameter that states to what degree the data has been anonymised. This parameter should not be kept secret, as it tells entities that are part of a dataset how well, and in what way, their privacy is being preserved. Take something like k-anonymity: the k tells you that every equivalence class in the dataset has a size >= k, i.e. for every entity in the dataset, there are at least k-1 other identical entities in the dataset. There are a lot of things wrong with k-anonymity, but at least it's transparent. Tech companies however just state in their Privacy Policies that "[they] care a lot about your privacy and will therefore anonymise your data", without specifying how they do that. 2. Sharing anonymised data with other organisations (this is called Privacy Preserving Data Publishing, or PPDP) is virtually always a bad idea if you care about privacy, because there is something called the privacy-utility tradeoff: you either have data with sufficient utility, or you have data with sufficient privacy preservation, but you can't have both. You either publish/share useless data, or you publish/share data that does not preserve privacy well. You can decide for yourself whether companies care more about privacy or utility. Luckily, there's an alternative to PPDP: Privacy Preserving Data Mining (PPDM). With PPDM, data analysts can submit statistical queries (queries that only return aggregate information) to the owner of the original, non-anonymised dataset. The owner will run the queries, and return the result to the data analyst. Obviously, one can still infer the full dataset as long as they submit a sufficient number of specific queries (this is called a Reconstruction Attack). That's why a privacy mechanism is introduced, e.g. epsilon-differential privacy. With e-differential privacy, you essentially guarantee that no query result depends significantly on one specific entity. This makes reconstruction attacks impossible. The problem with PPDM is that you can't sell your high-utility "anonymised" datasets, which sucks if you're a big boi data broker.
- amelius 5y ago
  Can advertisers be legally forced to use these mathematical techniques? Can we perhaps have a trusted third party which anonymizes data for these companies?
  - 5y ago
specialist 5y ago
Two tangential "yes and" points: 1) I'm not smart enough to understand differential privacy. So my noob mental model is: Fuzz the data to create hash collisions. Differential privacy's heuristics guide the effort. Like how much source data and how much fuzz you need to get X% certainty of "privacy". Meaning the likelihood someone could reverse the hash to recover the source identity. BUT: This is entirely moot if original (now fuzzed) data set can be correlated with another data set. 2) All PII should be encrypted at rest, at the field level. I really wish Wayner's Translucent Databases was more well known. TLDR: Wayner shows clever ways of using salt+hash to protect identity. Just like how properly protected password files should be salt+hash protected. Again, entirely moot if protected data is correlated with another data set. http://wayner.org/node/46 http://wayner.org/node/46 https://www.amazon.com/Translucent-Databases-Peter-Wayner/dp/0967584418 https://www.amazon.com/Translucent-Databases-Peter-Wayner/dp... Bonus point 3) The privacy "fix" is to extend property rights to all personal data. My data is me. I own it. If someone's using my data, for any reason, I want my cut. Pay me.

190 points by nemoniac 5y ago

dang 5y ago

We've merged https://news.ycombinator.com/item?id=29734713 https://news.ycombinator.com/item?id=29734713 into this thread since the previous submission wasn't to the original source. That's why some comment timestamps are older.

jmatthews 5y ago

Without digging in to details, I just want to agree that reversing into a clients PII, while not trivial, is certainly possible, and depending on the class of data, one can achieve fidelity levels in the high 80 percentiles.

lb1lf 5y ago

-Anecdotally, at a former employer we had an annual questionnaire used to estimate how content we were. The results, we were assured, would only be used in aggregate after having been anonymized. I laughed quite hard when the results were back - the 'Engineer, female, 40-49yrs, $SITE' in the office next door wasn't as amused. All her responses had been printed in the report. Sample size: 1.

omegalulw 5y ago
Isn't this a trivial example though? You require bucket size >= 10 say. Full on differential privacy is meant to address harder problems. For example, if you assume that there was a minor bug and people were asked to take the mandatory yearly survey again in a week. Now if you assume that the responses of people who responded before will stay the same when they re-respond, then by taking the difference between both results you can identify some specific people by looking at which bucket counts changed - this would be how the new guy who joined in that week voted. Differential privacy techniques are meant to anonymise data in way that you can't even do this.
- musingsole 5y ago
  It's trivial to fix in retrospect, nearly impossible to anticipate all similar corner cases ahead of time, and -- most importantly -- indicative of the type of logical bugs our systems of the world have that permit hacking from script-kiddies to state actors.
  - saghm 5y ago
    It may be nearly impossible to anticipate _all_ corner cases, but come on, GP was talking about there only being a sample size of 1; this is super easy to notice and then just not include in the results.
sqrt17 5y ago
At our (fairly large) company, you can query by team (and maybe job role) but it will hide responses where the sample size is smaller than a set number (I think 8 or 10). So yes it can be done but people have to actually care about it. The cautionary tale about k-anonymity (from Aaron Schwartz's book I think) is when the behavior of aggregates is also something that should be kept privates - the example was that the morning run of an army base in a foreign country was revealed because enough people did this with their smartwatches on that it formed a neat cluster.
- teraku 5y ago
  In Germany (I think all of the EU), a dataset can only be published if the sample size is at least >=7.
  - williamtrask 5y ago
    Just fwiw sample size isn’t a robust defence against this kind of attack. Check out Differential Privacy.
- ocdtrekkie 5y ago
  More details on the Strava Run incident: https://www.bbc.com/news/technology-42853072 https://www.bbc.com/news/technology-42853072
- pfraze 5y ago
  Isn’t location data particularly easy to de-anonymize? I remember reading some research that because people tend to be so consistent with their location, you could deanonymize most people in a dataset with 3 random location samples through the day
  - morganherlocker 5y ago
    This is likely the paper you are referring to: https://www.nature.com/articles/srep01376 https://www.nature.com/articles/srep01376 It states that 95% of people can be identified from just 4 location samples.
AstralStorm 5y ago
Even without sample size having these aggregated results makes it very easy to predict who picked what with a modicum of extra information. (Even silly binned personality type.)
rotten 5y ago
Never answer those honestly. More than half the time they aren't there to "help management understand how to do better", rather to purge people who aren't happy. "We don't need employees who don't love it here."
- lb1lf 5y ago
  -Oh, we were already deemed beyond redemption - we'd been a small company, quite successful in our narrow niche, only to be bought up by $MEGACORP. That was a culture clash. Big time. Nevertheless, I thought the same thing you did and filled in my questionnaire so that my answers created a nice, symmetrical pattern - it looked almost like a pattern for an arts&crafts project...
- vorpalhex 5y ago
  I always say I am unhappy and would take another offer in a heatbeat, and so far that strategy has worked well for me - I usually get offered a good salary bump amd bonus every year. Obviously YMMV.
AlexTWithBeard 5y ago
Can confirm. I used to have a team and at some point they all had to submit their feedback on my performance. The answers were then fed back to me unattributed, but it was pretty obvious who wrote what.
- vanusa 5y ago
  And - were there "consequences" for these people?
  - AlexTWithBeard 5y ago
    No consequences. There was nothing I didn't know before. EDIT: I first wanted to say that in any way there would be no consequences, but "there will be no consequences no matter what people write about me" is probably too strong of a statement for anyone to make.
    - ipaddr 5y ago
      Were there positive comments you didn't expect?
      - AlexTWithBeard 5y ago
        Kinda. I expected some people to leave a good feedback, but theirs was way beyond my expectations.

benreesman 5y ago

One click removed from an original source that is soaked in second-rate adtech crap. To my dying day I will regret being one of the architects of this insidious mechanism. The problem with trying to evade, or defeat, or even sidestep this stuff is that latent-rep embeddings break human intuition in their effectiveness. There was a time when the uniqueness of one’s signature could move money. Those pen twitches are still there to see in what order you click on links.

jaytaylor 5y ago
Having a conscience is good, but don't be too hard on yourself old friend.
kurthr 5y ago
Just wait until they have your unconscious eye-twitches. Foveal rendering is required for adequate resolution/refresh of VR/AR due to the bandwidths and GPU calcs involved (e.g. 6kx6kxRGBx2eyesx120Hz=~200Gb/s). Updating only the 20-30deg around the eye's focus allows reducing this by >10x which reduces power/weight and GPU cost dramatically.

aledalgrande 5y ago

To me it is crazy selling that data is even legal.

alkonaut 5y ago
To me it’s crazy that people go to work doing this. Even if you can negotiate the moral issues, how to people find things like adtech interesting? Some people seem legitimately excited when saying their product helps customers show “relevant experiences” and so on.
- shadowgovt 5y ago
  Because human behavior is absolutely fascinating, as is the fact it can be shaped.
- 1vuio0pswjnm7 5y ago
  If I am not mistaken, yesterday there was a top comment that started "Adtech veteran here." Perhaps surveilling people on the internet distorts one's sense of time, or, alternatively, some folks will just say anything to defend what they are doing.
- analognoise 5y ago
  "It is difficult to get a man to understand something when his salary depends upon his not understanding it." Upton Sinclair
- chupy 5y ago
  Working in adtech was interesting for me because I learned how to work with distributed systems, thought me the importance of optimizing, scaling, instrumentation etc. It's extremely satisfying shaving a ms or two in response times in an app when you have 15 ms in total to respond. I built for example a data pipeline that ingests Bluekai/Oracle data cloud data and from a big data perspective it's an amazing experience: from dealing with ingesting billions of records per day to making sure you can build audiences based on those records in near real time. From a privacy perspective it's very scary stuff. When you look at what data Bluekai has, there are hundreds of millions of profiles scattered around hundreds of thousands of segments. Basically you have segments from frozen food shoppers to people that bought a mercedes a class model 2020. P.S.I can say also the same thing about Google, FB etc. A lot of people that work there are basically working in adtech, especially in Google.

MrDresden 5y ago

Anecdotally, the only time I've seen a truly anonymized database was in a european genetics research company, due mainly to the rightly high amount of regulation required in the medical field. There was a whole separate legal entity, with its own board, that did the phenotype measurement gathering and stored the data in a big database on premise. The link between those measurements and the individual's personal identifiable record was then stored in a separate airgapped database which had cryptographic locks implemented (on the data and physical access to the server) so accessing the data took the physical presence of the privacy officers of each of the two companies (the measurement lab and the research lab) and finally what I found at the time to be the unique move; a representative from the state run privacy watchdog. To be able to backtrack the data to a person, there was always going to be a need to go through the watchdog. Required, not just legally mandated. All of the measurement data that was stored in the database came from very restricted input fields in the custom software that was made on prem (no long form text input fields for instance, where identifying data could be put in accidentally), and there was a lot of thought put into the design of the UI to limit the possibility that anyone could put identifiable data into the record. For instance numerical ranges for a specific phenotype where all prefilled in a dropdown, so as to keep user key input to a minimum. Much of the data also came from direct connections to the medical equipment (I wrote a serial connector for a Humphrey medical eye scanner that parsed the results straight into the software, skipping the human element all together). This didn't make for the nicest looking software (endless dropdowns or scales), but it fulfilled its goal of functionality and privacy concerns perfectly. The measurement data would then go through many automatic filters and further anonymizing processes before being delivered through a dedicated network pipeline (configured through the local isp to be unidirectional) to the research lab. Is this guaranteed to never leak any private information? No, nothing is 100%. This comes damn near close to it, but ofcourse would not work in most other normal business situations.

Hendrikto 5y ago
> To be able to backtrack the data to a person, there was always going to be a need to go through the watchdog. The assumption in the deanonymization literature is that this data is unavailable. So no, you don‘t need to go through any watchdog.

sanxiyn 5y ago

Remember that 33 bits of entropy are enough to identify everyone. It may not be legally so, but any data with 33 bits of entropy is technically PII, and you should treat it as such.

kortex 5y ago
> any data with 33 bits of entropy is technically PII, Well yes, but actually no. I can run UUIDgen 33 times and put it on pastebin, via tor. That does not mean I have 128 peoples' worth of PII. In fact I have ~0 bits of PII. If I were to paste one of those numbers in this comment, now they are all linked to kortex, and however much entropy that username has (call it 32, this account is leaky). But you are still short 127 "PIIs". The rest of the entropy means nothing. Conversely, a US phone number is <24 bits, but 100% PII. It's not 33 bits of "entropy". It's 33 bits worth of a distribution which can be correlated to an identity. If it's not correlated, it's not PII.
konschubert 5y ago
That makes no sense, sorry. Ok, 2^33 > world population, but that doesn't mean that the string "Hello world" is PII.
- AstralStorm 5y ago
  Unless someone is actually called Hello World. Or perhaps Bobby Tables. ;)
- halhen 5y ago
  That depends on the encoding, does it not? The binary sequence equal to ASCII "Hello world" might well be PII with many different encodings. By accident, of course, but nevertheless 33 bits of information would be enough.
- stillicidious 5y ago
  33 bits of entropy, not just 33 bits
  - deleted 5y ago

agsnu 5y ago

If you're interested in this topic, I recommend the chapter on Inference Control in Ross Anderson's excellent book "Security Engineering". It's one of the ones which is freely available on his web site https://www.cl.cam.ac.uk/~rja14/Papers/SEv3-ch11-7sep.pdf https://www.cl.cam.ac.uk/~rja14/Papers/SEv3-ch11-7sep.pdf

lrem 5y ago

In Google we have a bunch of researchers on anonymity and that whole thing is _hard_. I vaguely remember supporting, a couple years ago, a pipeline where logs stripped out of "all PII" came at one end, aggregated data out of the middle... Into an anonymity verifier, that then redirected much of it into /dev/null, because a technique is known to de-anonymise it. And the research on differential anonymity advanced quite a bit in the meantime.

geoduck14 5y ago
Did that stuff work?

float4 5y ago

Two things I'd like to say here 1. All anonymisation algorithms (k-anonymity, l-divergence, t-closeness, e-differential privacy, (e,d)-differential privacy, etc.) have, as you can see, at least one parameter that states to what degree the data has been anonymised. This parameter should not be kept secret, as it tells entities that are part of a dataset how well, and in what way, their privacy is being preserved. Take something like k-anonymity: the k tells you that every equivalence class in the dataset has a size >= k, i.e. for every entity in the dataset, there are at least k-1 other identical entities in the dataset. There are a lot of things wrong with k-anonymity, but at least it's transparent. Tech companies however just state in their Privacy Policies that "[they] care a lot about your privacy and will therefore anonymise your data", without specifying how they do that. 2. Sharing anonymised data with other organisations (this is called Privacy Preserving Data Publishing, or PPDP) is virtually always a bad idea if you care about privacy, because there is something called the privacy-utility tradeoff: you either have data with sufficient utility, or you have data with sufficient privacy preservation, but you can't have both. You either publish/share useless data, or you publish/share data that does not preserve privacy well. You can decide for yourself whether companies care more about privacy or utility. Luckily, there's an alternative to PPDP: Privacy Preserving Data Mining (PPDM). With PPDM, data analysts can submit statistical queries (queries that only return aggregate information) to the owner of the original, non-anonymised dataset. The owner will run the queries, and return the result to the data analyst. Obviously, one can still infer the full dataset as long as they submit a sufficient number of specific queries (this is called a Reconstruction Attack). That's why a privacy mechanism is introduced, e.g. epsilon-differential privacy. With e-differential privacy, you essentially guarantee that no query result depends significantly on one specific entity. This makes reconstruction attacks impossible. The problem with PPDM is that you can't sell your high-utility "anonymised" datasets, which sucks if you're a big boi data broker.

amelius 5y ago
Can advertisers be legally forced to use these mathematical techniques? Can we perhaps have a trusted third party which anonymizes data for these companies?
- 5y ago

specialist 5y ago

Two tangential "yes and" points: 1) I'm not smart enough to understand differential privacy. So my noob mental model is: Fuzz the data to create hash collisions. Differential privacy's heuristics guide the effort. Like how much source data and how much fuzz you need to get X% certainty of "privacy". Meaning the likelihood someone could reverse the hash to recover the source identity. BUT: This is entirely moot if original (now fuzzed) data set can be correlated with another data set. 2) All PII should be encrypted at rest, at the field level. I really wish Wayner's Translucent Databases was more well known. TLDR: Wayner shows clever ways of using salt+hash to protect identity. Just like how properly protected password files should be salt+hash protected. Again, entirely moot if protected data is correlated with another data set. http://wayner.org/node/46 http://wayner.org/node/46 https://www.amazon.com/Translucent-Databases-Peter-Wayner/dp/0967584418 https://www.amazon.com/Translucent-Databases-Peter-Wayner/dp... Bonus point 3) The privacy "fix" is to extend property rights to all personal data. My data is me. I own it. If someone's using my data, for any reason, I want my cut. Pay me.