4 ms·
Speaking from the Devil's Advocate side, as I work in Higher Education analytics: First: this data is, by the vast majority of institutions, used very respectf
by ineedasername 7y ago
Speaking from the Devil's Advocate side, as I work in Higher Education analytics:
First: this data is, by the vast majority of institutions, used very respectfully in terms of privacy rights and it is ABSOLUTELY never sold. Third part vendors that provide hosted systems are REQUIRED to affirm that they pass nothing on to any other party whatsoever. (I should note that this is the case in public and traditional private non-profit schools. For-profit schools are much less scrupulous and should not be regarded as trustworthy in this respect)
Beyond this, students are apprised of these uses of data with the ability to opt out. This is done as upfront as we can, very "in your face", though the tendency to click "agree" to something without reading, even very short and to the point, is of course always an issue.
Finally, the data collected is extremely useful. I cannot emphasize that enough. It is extremely useful in helping students be more successful. And it's not used in terms of surveillance. It's used pretty much exclusive in understanding the patterns the coincide with success and those that coincide with failure, dropout, academic struggling, and so on. In short, it is used to proactively intervene with students at risk rather than wait for it to be too late, e.g., final grades are in and no tutoring intervention is possible. A recent use had me using Xgboost to examine patterns of LMS usage to determine students at risk of early drop out. It worked, and we can now identify about one third of early drop outs with practically no effort and almost no false positives and work with those students to address there problems before they go too far.
I'm sure privacy maximalists will not be satisfied with the above, but really, within the realm of data comprehensive I idividual data collection, I'm not aware of any business, much less an entire industry, that treats such data as responsibly as this, and that really is concerned with using it only to further the best interests of it's constituents. Obviously I'm biased. That probably means things are not as rosy as I envision. But I think it is nonetheless an important and valid synopsis of the issue.
- wott 7y ago> It worked Of course, it works. And it would work even better if you permanently installed an anal probe in each student, plus a few electro-stimulators, and an automatic feeder.
- ineedasername 7y agoThat is an intellectually dishonest extension of this to the absurd. If you wish to discuss the merits or faults of this I will do so. This is not some clear cut issue. I know there are privacy concerns. We try to handle them as transparently and honestly as possible. In that effort, we save students millions of tuition dollars and years of their lives. I am receptive to arguments that the privacy tradeoff isn't worth it, we have that discussion every time we examine possible data sources and have decided not to use some. But your comment is beneath any ability to have a reasonable discussion on the topic.
- zbentley 7y agoThat's a super uncharitable interpretation of the GP's argument. Privacy is a spectrum. By your "logic", should students' entire identities be concealed from education personnel? Of course not. That's just as ridiculous as what you proposed.
- edflsafoiewq 7y agoOn the contrary, it's a concise rejection of the worldview that privacy means privacy from others, not privacy from me, and that students are interchangeable units subject not to any consideration of ethics but only performance.
- zbentley 7y agoWhere is that worldview evidenced in the post you were responding to? The poster seemed extremely cognizant of privacy concerns, and was defending a limited collection of specific data under very narrow circumstances for similarly delimited purposes. Put another way, saying "I think some specific data collection is ethical" is not the same as saying "all data collection I do is ethical, others may not be ethical". They also seemed to implicitly reject the students-are-interchangeable idea: the whole point of gathering this data is to determine if students (as individuals, classes, or entire schools) are struggling or in need of help. That's the opposite of interchangeable; that's finding suffering outliers. I'm harping on it 'cause I think I may be closer to your position on this issue than I am to the original poster's, but ... you're making extremist straw men, talking about feeding trays, and all but reciting the lyrics from the Pink Floyd song. I think that's making pro-privacy positions look bad. EDIT also what about my first point: is any data collection on students ethical? Names? Addresses? Test results? Or should teachers only operate behind double-blind, voice-and-handwriting-obfuscated screens from their students? See? I can make straw men, too.
- jdkee 7y ago“ respectfully in terms of privacy rights and it is ABSOLUTELY never sold.” Could it be hacked? How good are university IT security operations?
- ineedasername 7y agoAt my school, they're surprising good. I mean, the systems aren't in an air gap, but we use 2FA, requests for data access are scrutinized, and not from a bureaucratic perspective but from a conversational perspective: "Let's talk about what you're trying to do with this data, what's the minimum needed for your goal? Is on-going access really necessary or can we provide a single extract that you access via a secure environment?" That sort of thing. Infrastructure security isn't something I'm involved with, but we do have a robust security team that works on it, that I engage with when working through vendor checklists and other security matters, and they are up to date on the field and conscientious to the point of (reasonable) paranoia. We're also not a high-profile target: Most hackers looking to crack a system holding personal data seem most concerned with high value targets with millions, not thousands of records. That doesn't stop our security team from treating the possibility with the utmost seriousness though. However, the above applies to where I work. This is not universally the case. I think most schools are increasingly aware of the potential, are take minimally necessary steps to make sure they aren't wide open, but there are simply too many small schools with 1,000 to 5,000 students that simply don't have the scale needed to really devote resources to this. Again, the low-value target helps here, but it's not great.
- crankylinuxuser 7y ago> We're also not a high-profile target Really now? Have you checked your school on Shodan recently? Long story short, there's enough people doing hacking, with varied enough interests. You don't know what makes someone else tick.
- zbentley 7y agoThanks for the respectful and nuanced take. I am somewhat concerned with where this stuff leads, not because I assume it's a one step process between education analytics and oppressive dystopia, but precisely because the use cases you describe seem like great ideas. I agree that minimizing drop-out rates is important, and that we should avail ourselves of whatever tools we can to do it. What you're describing does not bother me, and seems like a reasonable use of the data. So why not take it a step further? Let's authorize a limited release of student-identifying data to government social work agencies (like the DHS in the United States), so that concerning student performance data could be correlated with whether a student's family was on income/food/housing assistance. Heck, to maximize the signal, we could even correlate it with families who aren't on assistance but whose homes are likely to be foreclosed upon (leading to a likely student move, possibly changing schools--a very disruptive educational experience whose impact can be mitigated with careful preparation). Why not correlate with criminal records, to help promote after-school programs to students who would otherwise be returning to troubled or dangerous homes? To be clear, I'm bothered because those things all seem like good ideas. I think they'd probably work, if done properly. And would probably help a lot of people. The systems we have currently to try to accomplish those things (overworked social workers, underfunded public-education and after-school programs) aren't succeeding. ...they'd also result in a de facto surveillance state. If that data were compromised by or sold to an outside (nongovernment) entity, the damage due to the breach would be very severe, from inability to get financing (because lenders know details about your life that may not be reflected on your credit score) to individual persecution (e.g. employer discrimination against family members whose leaked data indicated their financial/housing situation may put their long-term employment at risk). If the government programs using that data were "compromised" (i.e. staffed, perhaps via an electoral sea-change, with discriminatory people), the damage would be even worse. School funding allocation could suffer because someone at the top decided "Our data shows that so many of the students at $school_x have so much else going against them in their lives--why bother? Starve that school for funding and let it wither; that money would be better allocated to schools attended by our kids/kids of different races/kids with higher income." Additionally, correlations could be dishonestly interpreted by bigoted officials ("the algorithm says that female/male/dyslexic/$race_x kids are a lost cause!"). There are other, logistical issues. If school staff (i.e. counselors) have direct access to all this information, could they discriminate against kids and their families if bigoted? If school staff didn't have direct access, will the right conclusions be drawn by distant analysts (or worse, algorithms) without the personal context of counselors and teachers? A failure to consider all those factors could result in IEP-like things that harm, rather than help (consider: "You talked back to counselor Bob, so he won't authorize you for additional meal points or recommend you for after-school tutoring in an area where you're struggling", or "the DOE algorithm decided you need to take more English classes even though counselor John has been working on getting you individual reading time because he knows there's a bullying risk from other kids in the English program"). Many of these things are already happening, in varying degrees. I worry that making ubiquitous data available could make them worse, if it is not done carefully. I am also sensitive to the benefits that education analytics (and more pervasive, cross-cutting data availability for government use across the board) can provide. I think the answer is to proceed carefully (i.e. the opt-out buttons you discuss, and careful data anonymization whenever it makes sense to do so) and to insist on comprehensive regulation regarding data exchange between government programs, and between those programs and non-government entities. EDITS: explained logistical issues. Typos.
- cmcd 7y agoJust because the institution does not use the data irresponsibly doesn't mean the service provider will not. Not to mention third parties which could potentially gain access to this data directly or indirectly by listening for the check in signal devices send. Per the article, this is not being used to help the students succeed, instead it is being used to damage their grade for a lack of attendance. I think most reasonable people would agree attendance is not a reliable metric by which to base a students grade. The exception being discussion based or interactive courses like debate or public speaking.
- ineedasername 7y agoWe have extremely stringent security checklists for 3rd part vendors, as I alluded to in my post. Both the vendors, and any of the vendor's vendors, must be able to demonstrate that data is not used for any purpose but the school's. As for attendance, automated attendance tracking has been around for decades in the form of swipe cards, and simply automated a process professors undertook manually. No data is captured that would not have been in the analog world, and professors are still free to have no attendance policies at all schools I have spoken to about it. Granted, that may only be a sample size of 100 or so.
- jahmed 7y agoI work in higher ed and we use a student success analytics platform. Where I see this going wrong is administrators want a system to do everything. We want to identify the at-risk student, nudge them automatically, and have a successful outcome. Schools don't want to invest in people to actually make outreach or connection with students because that is hard. As a result, we generate a ton of data with little to show other than numbers. I'd be very curious to learn about your LMS usage project. Are you saying you identified at-risk students quickly enough to intervene at a point in the term where it made a difference between passing and failing and that the school successfully intervened in those cases?
- ineedasername 7y agoWe're not perfect, but as we have brought on analytics systems and engaged in institutions drive projects like my use of Xgboos with LMS data, we have invested heavily in additional advising resources because we recognize that data is irrelevant if you don't have people that will take action on it. I can make data driven predictions all day long about students at risk. What matters is what do we do about it? I've spoken with schools that thought buying an analytics platform was all they had to do. They have universally been disappointed with their lack of results.
- jahmed 7y agoThat's great, thanks. I get frustrated as a technical staff person, who personally enjoys being on the cutting edge, often being the one in meetings saying 'data is great but how will we actually use it?'. Honestly, I'll probably retire form higher ed in a few years and go work for an outfit like yours.
- ineedasername 7y agoHah, actually the "outfit like mine" is a university that has relatively recently come to some very good conclusions on this sort of thing. 5 or 6 years ago... Not so much. But one nice thing about working for an institution like this is that mostly people are looking to do the right thing for students, and don't just look at the thing as some commercial revenue extraction machine.
- mirimir 7y ago> Beyond this, students are apprised of these uses of data with the ability to opt out. What happens if they opt out? And can they opt out of assessment by default, while still allowing data collection? That is, could the data be collected, but go into a secure archive, which only the student could access? So if the student isn't doing well, they could release selected data to advisers.
- ineedasername 7y agoUnfortunately it's not as granular as that, so there is still lots of room for improvement. There are some specific things that can be opted out of as required by FERPA, but otherwise it's a bit all or nothing.
- ben_jones 7y ago> First: this data is, by the vast majority of institutions, used very respectfully in terms of privacy rights and it is ABSOLUTELY never sold. Bull shit. Bull shit. Bull shit. Ethical data usage is similar to end-to-end encryption in that it requires rigour at every transfer point, competency and expertise in its implementation, and thorough third party auditing. Are you telling me universities represent any of this?
- LMYahooTFY 7y agoI think I understand the rationale behind this fairly well, and find it sensible to a certain degree. With that said, I find a particular part of your explanation more intellectually dishonest than the anal probe comment. That's not to say you're being flippant as perhaps that commenter was. To claim that you're not using this data in terms of surveillance is utterly nonsensical. If you can demonstrate to me how you're not actively conducting surveillance, I'm open to hearing it. Privacy trade offs may provide measurable gains, but I've yet to come across even attempts at quantifying the loss of individual privacy. You can show asymptotically improving trends, but the costs seem to remain blank, unknown, and I think that is what our flippant friend was trying to get at.