5 ms·

Not if you expand what's considered sensitive!

by dak1 8y ago

mooman219 8y ago
I'd always want to make sure no PII is accidentally leaked. Example: in 1997, researchers at MIT showed that using only gender, date of birth and ZIP code, it is possible to identify the majority of US residents! They proved their point by identifying the Massachusetts governor's medical records in a publicly-available dataset that was presumed anonymous. In 2010, Netflix published an “anonymized” dataset of movie ratings by users. After it was released to the public, researchers were able to identify many Netflix users, even though the dataset only contained user ID, movie, rating, and rating date.
- AckSyn 8y ago
  Of course they used identifying information to actually identify people. This is 101 level of forensics and research.
- aaomidi 8y ago
  Meh. Voter records are all basically public and most have full name, address, phone number and even sometimes party affiliations. I'd argue that keeping those public is also a net good for society.
  - skookumchuck 8y ago
    There are two issues in tension here. One is a right to privacy, but the other is a right to audit who is actually voting. If the voter rolls were secret, there's nothing to stop a massive fraud by those in power to stay in power.
    - mattferderer 8y ago
      Party affiliation should not be public record. I would like to understand why it is.
      - hndamien 8y ago
        Kind of defeats the purpose of a public ballot.
        anth_anm 8y ago
        No, you can have the lists of voters without having any information about who they voted for (or in this case, party affiliation which gives who they probably voted for).
        hndamien 8y ago
        Wow! Did I write "public" - I meant "secret" - it's been a long day. Of all the comments I've made that should have been down voted, the one I made above was it. HN gods are having mercy on me it seems.
        skookumchuck 8y ago
        In the US we have a secret ballot. Some effort is gone to to not collect information about what you voted for, only that you voted. Countries with sham elections tend to be open ballot, as the voters need to show they support the current ruler. This is why the selected candidate wins with 98% of the vote or some such. You can't even get 70% of people to agree on which direction the sun rises. A secret ballot is necessary for free elections.
      - nradov 8y ago
        It's largely for primary voting in states that have partisan primaries. If you're registered as a Democrat then you receive a ballot with only the Democratic candidates. Of course one could argue that all primaries should be non-partisan. Or that governments shouldn't spend money on running partisan primaries on behalf of political parties which are private organizations.
        mattferderer 8y ago
        I understand having to select a party for primaries & the pros & cons of it. That is a different & even more complicated issue. I just don't understand why which party I voted for needs to be "public" record. I don't believe that companies & the general public need to know. I've used campaign software that shows how incredibly easy it is to select a list of people & market towards those who voted one way or another in the primary.
  - tracker1 8y ago
    It's funny, I work for an election services company, how much concern (hard software requirements) there are to keeping said public information secure. Even more funny are when specifications for engines for weapon systems (tanks, aircraft, not the guns/missiles, etc) are tightly guarded secrets domestically but considered public information overseas.
- anth_anm 8y ago
  It's not just PII, it's anything the government just doesn't want people to know. For whatever reason. I'm just pointing this out because I'd like to see a movement towards actually more open government.
- tvanantwerp 8y ago
  Where I work, a tax policy think tank, we purchase from the IRS an anonymized data set of about 100k sample tax returns that we use for modeling the effects of changes to tax policy. We've got an agreement with the IRS that we can never share that dataset, and I'm pretty sure this is why. While I'd love to be able to make our tax model results more transparent, the risk of de-anonymization is too high. I think another tax policy group is trying to create a synthetic dataset that is close to the sample in terms of outputs but is entirely made up, so that it could be used for verification of results by third parties--I hope they succeed.
- mmmmmmmmm 8y ago
  You've been doing some bathroom reading I see.
- brokenmachine 8y ago
  > researchers at MIT showed that using only gender, date of birth and ZIP code, it is possible to identify the majority of US residents! Source? How were they able to do that?
  - maxerickson 8y ago
    There are large databases that have names, zipcodes and dates of birth in them. If you include year, birth dates carry an awful lot of information.
  - Someone 8y ago
    2 genders times ~25,000 date of births (70-ish years) times 40,000 zip codes equals 2 billion. The population won’t be uniformly distributed over date of birth or zip code, but that gives a factor of six or so leeway. The 87% paper is https://dataprivacylab.org/projects/identifiability/paper1.pdf https://dataprivacylab.org/projects/identifiability/paper1.p.... It says: ”It was found that 87% (216 million of 248 million) of the population in the United States had reported characteristics that likely made them unique based only on {5-digit ZIP, gender, date of birth}. About half of the U.S. population (132 million of 248 million or 53%) are likely to be uniquely identified by only {place, gender, date of birth}, where place is basically the city, town, or municipality in which the person resides. And even at the county level, {county, gender, date of birth} are likely to uniquely identify 18% of the U.S. population. In general, few characteristics are needed to uniquely identify a person.” More info likely available via http://latanyasweeney.org/work/identifiability.html http://latanyasweeney.org/work/identifiability.html (discussed in https://news.ycombinator.com/item?id=2942967 https://news.ycombinator.com/item?id=2942967)
- wodenokoto 8y ago
  What did they cross reference the IDs with for the Netflix data?
bilbo0s 8y ago
True. Kind of depends on what they say is "sensitive". Probably not much of a facility for appealing that designation either. I sorta look at it like the "First Step" prison reform act, it's only the start. Baby steps.

Not if you expand what's considered sensitive!

by dak1 8y ago

Not if you expand what's considered sensitive!

I'd always want to make sure no PII is accidentally leaked. Example: in 1997, researchers at MIT showed that using only gender, date of birth and ZIP code, it is possible to identify the majority of US residents! They proved their point by identifying the Massachusetts governor's medical records in a publicly-available dataset that was presumed anonymous. In 2010, Netflix published an “anonymized” dataset of movie ratings by users. After it was released to the public, researchers were able to identify many Netflix users, even though the dataset only contained user ID, movie, rating, and rating date.

AckSyn 8y ago
Of course they used identifying information to actually identify people. This is 101 level of forensics and research.
aaomidi 8y ago
Meh. Voter records are all basically public and most have full name, address, phone number and even sometimes party affiliations. I'd argue that keeping those public is also a net good for society.
- skookumchuck 8y ago
  There are two issues in tension here. One is a right to privacy, but the other is a right to audit who is actually voting. If the voter rolls were secret, there's nothing to stop a massive fraud by those in power to stay in power.
  - mattferderer 8y ago
    Party affiliation should not be public record. I would like to understand why it is.
    - hndamien 8y ago
      Kind of defeats the purpose of a public ballot.
      - anth_anm 8y ago
        No, you can have the lists of voters without having any information about who they voted for (or in this case, party affiliation which gives who they probably voted for).
        hndamien 8y ago
        Wow! Did I write "public" - I meant "secret" - it's been a long day. Of all the comments I've made that should have been down voted, the one I made above was it. HN gods are having mercy on me it seems.
      - skookumchuck 8y ago
        In the US we have a secret ballot. Some effort is gone to to not collect information about what you voted for, only that you voted. Countries with sham elections tend to be open ballot, as the voters need to show they support the current ruler. This is why the selected candidate wins with 98% of the vote or some such. You can't even get 70% of people to agree on which direction the sun rises. A secret ballot is necessary for free elections.
    - nradov 8y ago
      It's largely for primary voting in states that have partisan primaries. If you're registered as a Democrat then you receive a ballot with only the Democratic candidates. Of course one could argue that all primaries should be non-partisan. Or that governments shouldn't spend money on running partisan primaries on behalf of political parties which are private organizations.
      - mattferderer 8y ago
        I understand having to select a party for primaries & the pros & cons of it. That is a different & even more complicated issue. I just don't understand why which party I voted for needs to be "public" record. I don't believe that companies & the general public need to know. I've used campaign software that shows how incredibly easy it is to select a list of people & market towards those who voted one way or another in the primary.
- tracker1 8y ago
  It's funny, I work for an election services company, how much concern (hard software requirements) there are to keeping said public information secure. Even more funny are when specifications for engines for weapon systems (tanks, aircraft, not the guns/missiles, etc) are tightly guarded secrets domestically but considered public information overseas.
anth_anm 8y ago
It's not just PII, it's anything the government just doesn't want people to know. For whatever reason. I'm just pointing this out because I'd like to see a movement towards actually more open government.
tvanantwerp 8y ago
Where I work, a tax policy think tank, we purchase from the IRS an anonymized data set of about 100k sample tax returns that we use for modeling the effects of changes to tax policy. We've got an agreement with the IRS that we can never share that dataset, and I'm pretty sure this is why. While I'd love to be able to make our tax model results more transparent, the risk of de-anonymization is too high. I think another tax policy group is trying to create a synthetic dataset that is close to the sample in terms of outputs but is entirely made up, so that it could be used for verification of results by third parties--I hope they succeed.
mmmmmmmmm 8y ago
You've been doing some bathroom reading I see.
brokenmachine 8y ago
> researchers at MIT showed that using only gender, date of birth and ZIP code, it is possible to identify the majority of US residents! Source? How were they able to do that?
- maxerickson 8y ago
  There are large databases that have names, zipcodes and dates of birth in them. If you include year, birth dates carry an awful lot of information.
- Someone 8y ago
  2 genders times ~25,000 date of births (70-ish years) times 40,000 zip codes equals 2 billion. The population won’t be uniformly distributed over date of birth or zip code, but that gives a factor of six or so leeway. The 87% paper is https://dataprivacylab.org/projects/identifiability/paper1.pdf https://dataprivacylab.org/projects/identifiability/paper1.p.... It says: ”It was found that 87% (216 million of 248 million) of the population in the United States had reported characteristics that likely made them unique based only on {5-digit ZIP, gender, date of birth}. About half of the U.S. population (132 million of 248 million or 53%) are likely to be uniquely identified by only {place, gender, date of birth}, where place is basically the city, town, or municipality in which the person resides. And even at the county level, {county, gender, date of birth} are likely to uniquely identify 18% of the U.S. population. In general, few characteristics are needed to uniquely identify a person.” More info likely available via http://latanyasweeney.org/work/identifiability.html http://latanyasweeney.org/work/identifiability.html (discussed in https://news.ycombinator.com/item?id=2942967 https://news.ycombinator.com/item?id=2942967)
wodenokoto 8y ago
What did they cross reference the IDs with for the Netflix data?

bilbo0s 8y ago

True. Kind of depends on what they say is "sensitive". Probably not much of a facility for appealing that designation either. I sorta look at it like the "First Step" prison reform act, it's only the start. Baby steps.