5 ms·

So long as the data cannot identify specific students (i.e. it's sufficiently anonymized) what's the issue?

by nvy 3y ago

dragonwriter 3y ago
> So long as the data cannot identify specific students It almost certainly can, even if it does not explicitly do so.
chaps 3y ago
.....do you genuinely think that the data can be sufficiently anonymized to protect the privacy of minors?
- eimrine 3y ago
  I believe that the data can be sufficiently faked to be anonymized.
  - pjc50 3y ago
    That also renders it useless!
    - IX-103 3y ago
      That's not strictly true. There's some recent work (as fascinating as it is incomprehensible) on generating datasets that share most aggregate properties with the actual dataset (measured through joint probability distributions), but do not reveal more than some epsilon of information about any individual contained in the original data set. These have the potential to revolutionize private computation and analysis, as they provide provable hard (theoretical) limits on the amount of information you can learn about individuals regardless of the type of analysis performed on the proxy dataset.
      - mrguyorama 3y ago
        Two datasets that share many aggregate statistics are not interchangeable.
- doctorpangloss 3y ago
  I've actually gone through this process with the CDE and I was denied access. The privacy issue is a huge red herring, used to co-opt well meaning people like you. I requested data about STAR, the California standardized test used for Lowell's admissions. I wanted rows of the form (randomized student ID, STAR question ID, answered correctly), however they were recorded, and literally nothing more. They rejected the request because (1) they claimed such records didn't exist, which makes no sense because how exactly did they administer the test then; and (2) because standardized testing is carved out, in their opinion, from the related sunshine law. Why did I want these records? I wanted to show that scoring well on tests and using them to gate admissions doesn't mean what people think it means. Specifically, that if you administered the test Lowell used (STAR) by hardest question first, then terminated the test after the student gets N (close to 1) questions wrong, you would select nearly the same list of students. Only asking the vast majority of students only e.g. 1 question, which they all get wrong, can't possibly measure how much they study, how comprehensive their knowledge is, etc. But these claims are routinely made in defense of the test and its purpose in selecting a class. This is coming from someone who wants test based admissions. So clearly political, right? I had to carefully word my request around all these conclusions. If you read the CDE's requirements, they really have specific political goals. You either align with them or you don't. And I tried to work around that, and I stilled failed. They just looked at the absence of a political bent, and correctly concluded that it wasn't evidence of absence. If you want to do good, politically impactful educational research: run your own school. That's what the CDE wants you to do. It's not about discovering how to improve public schools.
  - chaps 3y ago
    Not sure it's fair to say it's a red herring, or that I'm "co-opted" like you suggest. Transparency is kind of my main dig -- I get it. Like, I recently helped a small team of researchers with some FOIA requests to get access to similar information you were denied. But at the end of the day it's fundamentally important to understand at what point transparency and privacy intersect.
    - doctorpangloss 3y ago
- mrangle 3y ago
  Data has been sufficiently anonymized for decades.
- nvy 3y ago
  The specific data? No idea, never seen it. Do I think that, in principle, data sets can be anonymized? Of course I do. Your incredulous tone and excessive ellipsis seems to imply you find this position to be ridiculous, so maybe you'd better be a little less snide and a little more expansive on, what, exactly, your problem is.
  - deleted 3y ago
    [deleted]
  - deleted 3y ago
    [deleted]
  - chaps 3y ago
    It's because their confidence in deidentifying data doesn't match the significant risk. If you think it's worth it and are willing to take that risk that's on you and those you risk harming.
giovannibonetti 3y ago
The issue is the "sufficiently anonymized" part. Given a large enough number of dimensions, you may be able to identify students well enough. For example, if you take all students that took course A at time X, course B at time Y, course C at time Z and so on, eventually you might be able to narrow it down to a very small group, perhaps to even a single student.
- chaps 3y ago
  Correct. I've done a fair amount of de-obfuscation work and it's frighteningly trivial sometimes.
  - arcticfox 3y ago
    How can the researchers simultaneously publish research and not be allowed to testify to their conclusions in litigation though? It seems clear that this is not a privacy concern and is rather a protective measure.
    - chaps 3y ago
      On that we both agree.
- godelski 3y ago
  This will also probably follow a power series too. So it isn't unlikely that you could deanonymize someone given just 2 courses. Not much information is needed to encode a lot of things.
- mrangle 3y ago
  You're attempting to make an argument against any anonymized data being used in research. You'd have to do better than a hypothetical to make headway with it. Moreover, the logic would have to carry over to the very common practice of anonymizing data in professional communications (like training). Which would have HIPAA implications for some students. The common anonymizing practices have been utilized for decades without privacy breaches of note. That track record is also what your argument would have to defeat.
mlyle 3y ago
Anonymizing a huge data set like this is impossible. Also, the burden of proof is on those that say that the data has no privacy implications, not on those who are like "ehhh, it's probably safe to release this."
- prepend 3y ago
  > Anonymizing a huge data set like this is impossible. That depends. The entire dataset of course because it’s everyone’s student records. But you can probably subset it to the extent that it’s still useful and perturb enough to protect individuals and be statistically equivalent. And you could also generate a bunch of aggregate results that do stuff like identify average grade differences before and after periods while correcting for other differences without including individual identifiers.
  - chaps 3y ago
    You're moving goalposts, friend. What you're suggesting and what OP are suggesting are in two completely different categories of disclosure extent. I don't think anybody here is suggesting that no data should be available to the public.
    - prepend 3y ago
      I don’t think so. OP was saying the data should be released. CA is saying it can’t at all for privacy reasons. I’m saying that the data OP can be released with perturbations made to protect privacy and the data are still useful. I don’t think anyone is calling for the raw data to be dumped. But for as much data as possible to be released.
      - mlyle 3y ago
        I agree with the person's criticism of your comment. Yes, obviously there is a level of aggregation where privacy concerns no longer hold. But there is no trivial transformation that allows education researchers the data they need but preserves anonymity. Education researchers want to aggregate and statistically sample the data in new ways; pre-aggregating it removes most ability to do so. If you want to do a principal component analysis of a few variables-- good luck with aggregate data. If you provide nearly any data at the student-level, there's a pretty high chance that it can be deanonymized. At the same time, the state's position of attempting to prevent education researchers from participating in litigation (when using only public, non-restricted data) is egregious.
amalcon 3y ago
The word "anonymized" needs to be excised from our collective vocabulary. "Anonymization" is not a thing that can be meaningfully done to a dataset about individuals. Coarse aggregation is possible, and the only practical way to achieve this end, but this has its own drawbacks in a research context.
jasonlotito 3y ago
Data aggregation does not guarantee privacy. Unless you can guarantee privacy, which "sufficiently anonymized" does not, then no.

So long as the data cannot identify specific students (i.e. it's sufficiently anonymized) what's the issue?

by nvy 3y ago

So long as the data cannot identify specific students (i.e. it's sufficiently anonymized) what's the issue?

dragonwriter 3y ago

> So long as the data cannot identify specific students It almost certainly can, even if it does not explicitly do so.

chaps 3y ago

.....do you genuinely think that the data can be sufficiently anonymized to protect the privacy of minors?

eimrine 3y ago
I believe that the data can be sufficiently faked to be anonymized.
- pjc50 3y ago
  That also renders it useless!
  - IX-103 3y ago
    That's not strictly true. There's some recent work (as fascinating as it is incomprehensible) on generating datasets that share most aggregate properties with the actual dataset (measured through joint probability distributions), but do not reveal more than some epsilon of information about any individual contained in the original data set. These have the potential to revolutionize private computation and analysis, as they provide provable hard (theoretical) limits on the amount of information you can learn about individuals regardless of the type of analysis performed on the proxy dataset.
    - mrguyorama 3y ago
      Two datasets that share many aggregate statistics are not interchangeable.
doctorpangloss 3y ago
I've actually gone through this process with the CDE and I was denied access. The privacy issue is a huge red herring, used to co-opt well meaning people like you. I requested data about STAR, the California standardized test used for Lowell's admissions. I wanted rows of the form (randomized student ID, STAR question ID, answered correctly), however they were recorded, and literally nothing more. They rejected the request because (1) they claimed such records didn't exist, which makes no sense because how exactly did they administer the test then; and (2) because standardized testing is carved out, in their opinion, from the related sunshine law. Why did I want these records? I wanted to show that scoring well on tests and using them to gate admissions doesn't mean what people think it means. Specifically, that if you administered the test Lowell used (STAR) by hardest question first, then terminated the test after the student gets N (close to 1) questions wrong, you would select nearly the same list of students. Only asking the vast majority of students only e.g. 1 question, which they all get wrong, can't possibly measure how much they study, how comprehensive their knowledge is, etc. But these claims are routinely made in defense of the test and its purpose in selecting a class. This is coming from someone who wants test based admissions. So clearly political, right? I had to carefully word my request around all these conclusions. If you read the CDE's requirements, they really have specific political goals. You either align with them or you don't. And I tried to work around that, and I stilled failed. They just looked at the absence of a political bent, and correctly concluded that it wasn't evidence of absence. If you want to do good, politically impactful educational research: run your own school. That's what the CDE wants you to do. It's not about discovering how to improve public schools.
- chaps 3y ago
  Not sure it's fair to say it's a red herring, or that I'm "co-opted" like you suggest. Transparency is kind of my main dig -- I get it. Like, I recently helped a small team of researchers with some FOIA requests to get access to similar information you were denied. But at the end of the day it's fundamentally important to understand at what point transparency and privacy intersect.
  - doctorpangloss 3y ago
mrangle 3y ago
Data has been sufficiently anonymized for decades.
nvy 3y ago
The specific data? No idea, never seen it. Do I think that, in principle, data sets can be anonymized? Of course I do. Your incredulous tone and excessive ellipsis seems to imply you find this position to be ridiculous, so maybe you'd better be a little less snide and a little more expansive on, what, exactly, your problem is.
- deleted 3y ago
  [deleted]
- deleted 3y ago
  [deleted]
- chaps 3y ago
  It's because their confidence in deidentifying data doesn't match the significant risk. If you think it's worth it and are willing to take that risk that's on you and those you risk harming.

giovannibonetti 3y ago

The issue is the "sufficiently anonymized" part. Given a large enough number of dimensions, you may be able to identify students well enough. For example, if you take all students that took course A at time X, course B at time Y, course C at time Z and so on, eventually you might be able to narrow it down to a very small group, perhaps to even a single student.

chaps 3y ago
Correct. I've done a fair amount of de-obfuscation work and it's frighteningly trivial sometimes.
- arcticfox 3y ago
  How can the researchers simultaneously publish research and not be allowed to testify to their conclusions in litigation though? It seems clear that this is not a privacy concern and is rather a protective measure.
  - chaps 3y ago
    On that we both agree.
godelski 3y ago
This will also probably follow a power series too. So it isn't unlikely that you could deanonymize someone given just 2 courses. Not much information is needed to encode a lot of things.
mrangle 3y ago
You're attempting to make an argument against any anonymized data being used in research. You'd have to do better than a hypothetical to make headway with it. Moreover, the logic would have to carry over to the very common practice of anonymizing data in professional communications (like training). Which would have HIPAA implications for some students. The common anonymizing practices have been utilized for decades without privacy breaches of note. That track record is also what your argument would have to defeat.

mlyle 3y ago

Anonymizing a huge data set like this is impossible. Also, the burden of proof is on those that say that the data has no privacy implications, not on those who are like "ehhh, it's probably safe to release this."

prepend 3y ago
> Anonymizing a huge data set like this is impossible. That depends. The entire dataset of course because it’s everyone’s student records. But you can probably subset it to the extent that it’s still useful and perturb enough to protect individuals and be statistically equivalent. And you could also generate a bunch of aggregate results that do stuff like identify average grade differences before and after periods while correcting for other differences without including individual identifiers.
- chaps 3y ago
  You're moving goalposts, friend. What you're suggesting and what OP are suggesting are in two completely different categories of disclosure extent. I don't think anybody here is suggesting that no data should be available to the public.
  - prepend 3y ago
    I don’t think so. OP was saying the data should be released. CA is saying it can’t at all for privacy reasons. I’m saying that the data OP can be released with perturbations made to protect privacy and the data are still useful. I don’t think anyone is calling for the raw data to be dumped. But for as much data as possible to be released.
    - mlyle 3y ago
      I agree with the person's criticism of your comment. Yes, obviously there is a level of aggregation where privacy concerns no longer hold. But there is no trivial transformation that allows education researchers the data they need but preserves anonymity. Education researchers want to aggregate and statistically sample the data in new ways; pre-aggregating it removes most ability to do so. If you want to do a principal component analysis of a few variables-- good luck with aggregate data. If you provide nearly any data at the student-level, there's a pretty high chance that it can be deanonymized. At the same time, the state's position of attempting to prevent education researchers from participating in litigation (when using only public, non-restricted data) is egregious.

amalcon 3y ago

The word "anonymized" needs to be excised from our collective vocabulary. "Anonymization" is not a thing that can be meaningfully done to a dataset about individuals. Coarse aggregation is possible, and the only practical way to achieve this end, but this has its own drawbacks in a research context.

jasonlotito 3y ago

Data aggregation does not guarantee privacy. Unless you can guarantee privacy, which "sufficiently anonymized" does not, then no.