9 ms·

U.S. universities, rich in data, struggle to capture its value, study finds

amelius 4y ago
University data is getting outdated very quickly. Large companies have taken over the role of data curators. I personally think this is wrong. Big tech companies should build hardware, while government agencies should use that hardware to curate our data. Like how it was near the beginning of the internet. Companies should be as far away from our data as possible.
- robertlagrant 4y ago
  I think the exact opposite: government should know as little as is required to provide its services, as should any company. If you imagine a company can do damage with your data, wait til a bad government has it.
  - deleted 4y ago
    [deleted]
jimlongton 4y ago
I can't access the original study however taking data, explicitly naming security cameras for example, to "use it or merge it with data from external parties such as publishers or public or private sector organizations" will surely not seriously degrade student and staff privacy, right? The rush to "exploit" data reminds me of the dot com hype. It's one thing to use available data to make more informed decisions about things from course content to building occupancy. It's quite another to rush towards total surveillance because of a Fear of Missing Out of "exploitable data".
- McSinyx 4y ago
  > degrade student and staff privacy That's the primary value to be captured right there! Their privacy is highly valued, monetarily: > authors contend that universities have been slower than organizations in other _economic_ sectors Education in the US is just pure business.
  - a9h74j 4y ago
    Make of it what you will, I just saw on the project censored website that protecting student data is seen by some as a major priority. The article seems more PII and cybersecurity focussed, rather than recognizing the threat of intentional monetization. https://www.projectcensored.org/ferpa-and-higher-ed-should-prioritize-the-safety-of-students-private-data/ https://www.projectcensored.org/ferpa-and-higher-ed-should-p...
    - nerdponx 4y ago
      This is about digital security, i.e. protecting data from unauthorized access and use. It says nothing about protecting students themselves from surveillance and abuse by their own institutions.
tomrod 4y ago
This is the type of article that makes me wonder: "therefore, what?" Yes, absolutely, universities are awash in data they do little to anything with outside of hyperfocused laboratories. I wonder if having central data management groups would be an enabler or a bottleneck, however.
- Dwolb 4y ago
  Therefore they can’t optimize anything. They have 0 visibility into any of the operations and therefore can’t derive any insights or install controls. They have no idea if: Some professors more effective teachers than others. Which days students buy more food. Whether taking a given prerequisite leads to better mastery of advanced material. If additional spend is correlated with better student outcomes. For people in tech businesses, it’s such a shocking thing to read it’s hard to fathom how any of the leadership at universities find this situation acceptable.
  - jimlongton 4y ago
    I didn't go to a prestigious US university but even mine had all this data and used it. In tech we like to think we are the only ones to "get" data. We are not. Teaching outcomes are evident from exam results, grants won, papers published etc and are routinely used to determine promotions, staffing and more. Food/footfall etc are monitored and predicted by the private catering companies on campus. Budget reviews are comprehensive and require yearly and greater analysis of university performance on a per department basis. All that doesn't even include competitive processes like ranking tables for universities, applications for grants, participation in international research groups etc etc.
  - miguelazo 4y ago
    > They have no idea if: Some professors more effective teachers than others. What are you even talking about? Literally every course at most universities end with an evaluation. Obviously, there are other problematic ways to measure effectiveness (publications). How much that data gets used is another question and varies widely by institution. “People in tech businesses” love to exaggerate the quality of the data they have and its usefulness, hence the Subprime Attention Crisis that is rapidly eroding their cash on hand as the Fed’s cheap money and QE dries up.
ghiculescu 4y ago
This article feels like it took hundreds of words to say nothing at all. The funniest part was the example of security cameras. What big breakthroughs are universities hoping to achieve with this data?
- mistrial9 4y ago
  some industry players want to normalize facial recognition and location tracking
  - jimlongton 4y ago
    I had to laugh at one of the authors being Christine L. Borgman. I'm sure all our data will be treated respectfully when fully integrated into their unaccountable profit-driven total surveillance system.
- nonrandomstring 4y ago
  They're being pushed very hard from the supply side while lacking much push-back internally due to dwindling expertise and leadership. This 4 parter I wrote for Techrights [1] and the Times article that seeded it [2] go into detail. [1] http://techrights.org/2022/12/28/andy-farnell-on-british-universities/ http://techrights.org/2022/12/28/andy-farnell-on-british-uni... [2] https://www.timeshighereducation.com/campus/eliminating-harmful-digital-technologies-universities-guide https://www.timeshighereducation.com/campus/eliminating-harm...
- deleted 4y ago
  [deleted]
- foreverCarlos 4y ago
  Security cameras in universities seem to be accomplishing… nothing. I am getting very frequent emails (2-3 per week) from one of the top US universities with blurry unusable screenshots of perpetrators stealing items, breaking in, assaulting students, etc. Makes me think - should they not invest into better security instead?
nonrandomstring 4y ago
There's a troubling tone behind this article. It's a prophecy that demands to be fulfilled. Starting with the premise "Data is useful", it proceeds to pick at all the ways we've failed to make it useful... and we're damned well going to make it useful if it kills us! Maybe, just maybe (for those that dabble in the sceptical, explanatory game we call science) it might be that "bare data" has little use within certain contexts... say those that by definition are on the cutting edge of knowledge and reality, best steered by imaginative vision of leading experts. Data driven market cybernetics may be very useful in some industries, such as ones with physical logistics, complex supply chains, rapidly shifting supply sources and demand sinks. But the primacy of "data uber alles" should not be one-size-fits-all.
- lukev 4y ago
  There's a cost/benefit curve here. Having data can undeniably be useful. The question is, how much ROI does the insights in the data provide, relative to the cost to build the organizational infrastructure capable of extracting these insights? And there's a bootstrapping problem, because often it's hard to calculate the potential ROI without actually going through the work of building the infrastructure to process the data.
  - nonrandomstring 4y ago
    Indeed, absolutely true. And furthermore, sometimes the value of data, and of possible other sources, doesn't emerge until you start analysis and then deductively see new collection opportunities. It's a to-and-fro process. > how much ROI does the insights in the data provide, relative to the cost to build the organizational infrastructure Having seen the workings of academia for a long time now I'd argue very little. But I think the problem is more subtle. A Heisenbergian problem. When data collection involves people you have a triple problem of intrusion, distraction, and ossification. the act of trying to create a "data driven culture" in some contexts simply kills what's there. The cost of this, in addition to the bootstrapping costs of sensors and analytics are paralysing. Academia, by its very nature, if properly functioning, demands to be dynamic. If it's good, it will not stand still long enough for you to look at it. And if you measure it "too hard" it will evaporate as all the good people who resent ossification leave.
lukev 4y ago
I have no doubt the problem is worse in academia, but to be quite honest nobody is really doing a good job at curating, governing and unifying data across large organizations (at least without extreme expense.) Just look at the treadmill of architectures designed to "solve" this problem: integrated databases to "data warehouses" to "linked data" to "data lakes" to "data fabric" to "data mesh." Organizations that succeed do so because they have the resources to prioritize data management, and do so at significant team size and expense. But any organization that doesn't (or can't) view data curation as first class, top-line budget isn't in a position to capture even a small part of the theoretical value of their data. Consider the "data mesh" architecture, the latest fad in this space. It basically assumes that not only do you have a dedicated data curation team, each data source also has it's own dedicated team of curators capable of productizing data sources. That kind of thing is simply out of reach for most organizations.
- le-mark 4y ago
  I lived through an acquisition in the teens that I feel was an illustrative story. An old financial services provider embarked on a massive, whole organization modernization effort. The new CEO went all in and actually made the necessary investment in staff and headcount, which was remarkable in itself. A big part of the effort involved providing customers with big data solutions and all the implied benefits that would entail. They jumped aboard the hype train with gusto. Each quarter progress was reported and margins improved with a clear upward trend. From the trenches it was clear that even with the investment made, it was still not enough to reach any of the goals. After a year of “modernization” the company was acquired, the buyer literally salivating over the coming profit margin increase and “synergies” in selling into related markets. After another year post acquisition it became clear that the benefits were simply out of reach, and the profits did not arrive. The acquiring company was left with the low margin business it had always been. Layoffs ensued as the buyer sought to salvage something from the transaction. Turned out the entire modernization and big data effort had been an elaborate bait and switch committed by the board and CEO of the acquired company, masterfully executed.
  - WrtCdEvrydy 4y ago
Eumenes 4y ago
We need more ad-tech startups for the university/higher education industry NOW
- a9h74j 4y ago
  One model: YT for persons without an adblocker. Or: This lecture brought to you by Pepsi.
- musicale 4y ago
  > We need more ad-tech startups for the university/higher education industry NOW I had the same thought - intrusive analytics and a focus on exploiting and monetizing client data seem to have damaged the web, made gaming less fun, and made everything user-hostile and creepy. Do we really want universities to go down that path?
pcrh 4y ago
Universities already waste vast sums of money of vice-provosts, middle management, administration, etc. There's no need to encourage them to start treating their mission as one of data mining in order to "capture" more value from the students.
- foreverCarlos 4y ago
  While I agree with your general skepticism, I also believe there are ways these data could benefit the students. Extracting value from operational data could mean modernizing curriculum, offering more office hours, etc.
  - civilized 4y ago
    I think most of the value of "data" is captured in empowering individual contributors to observe their working conditions and the impact of their actions and adapt their strategy in response. The power of the central office spreadsheet wielders is very secondary. (Which is of course not to say that spreadsheets are not valuable, as any teacher knows.)
    - foreverCarlos 4y ago
      In an ideal world - yes. But seeing how overworked professors and other staff members already are in top universities, I doubt this is top of mind for them. There are courses with hundreds of enrolled students, weekly homework, practical projects, etc. IMO somewhat centralized DS tools are more suited to handle this load than an individual contributor (a professor in this context).
- blthree 4y ago
  You have a point about the amount of money spent on admin at Universities, but I think you (and most commenters here) don't really realize how bad the data situation is at a lot of major universities. If we want to streamline admin overhead at public universities, there's an incredible amount of time and money wasted due to extremely inefficient use/access to data. I work as a BI dev at a large Big 10 school (50k students/35k faculty and staff). For all those students and staff, there are a grand total of 3 data engineers, who are responsible for all our central data warehouses, including DBAing our oracle and redshift DBs. Just to get a new (untransformed) table added to the data warehouse from a source takes 6+ months, and that's only if it's from a source with an existing integration. On top of that, there are at least 50 people I know of whose job is basically to produce one or two reports manually in excel every week, and this isn't even considering people in finance or accounting. I'm talking about simple things like how many active research grants do we have, or how many students have enrolled in certain courses. These are reports (and entire fte positions) that could easily be automated with a single SQL query. Speaking of SQL, outside of the data engineering team, there are only 4 or 5 people who know any SQL out of the 100 I know of in reporting/BI positions. The "advanced" data teams are using MS access as an ETL tool to pull together data for tableau reports. However, there are a lot of institutional issues that make fixing those problems difficult. For one, while we have a central IT dept, we also have about 10 individual college-level IT teams, which means that data isn't just in different databases, but on a whole separate network. For example, if I want to create a report on student faculty ratio, I need to connect to VPN 1 to export faculty data from redshift, then switch to VPN 2 to export student data from Oracle, then switch to VPN 3 so that I can upload both datasets to our depts SQL server. After all that I can finally write a SQL query to get a student faculty ratio. Oh and when we need to update that ratio in a month, I'll have to go thru the whole manual extract/load process again. Forget automating that, since the network teams have no incentive to allow any tunnelling or bridging from one network to another. I could rant about this all day, but I think it's fair to say that there are still a ton of low-hanging fruit inefficiency-wise at universities. If we could get universities to value their data more highly, maybe that wod have the additional effect of solving some of these problems and even be a net money saver.
zitterbewegung 4y ago
I was in a social network research laboratory. Some of the best people at the lab went into industry while the others did do post docs. The professor that worked at the lab also went to Microsoft Research. Honestly if you were to go the route of being a professor or from MANGA what would you choose? Also, I think there are laws and also rules within Universities that would preclude the study to capture value and even then you would have to hire data analysts that would be external to being a professor so I think it would be hard to compete with MANGA in that regard also.
carbocation 4y ago
Oh good - an opportunity to use "begging the question" for its original meaning. I don't see a single example of "success" in the article. (They said "We unexpectedly found a pervasive void of infrastructure thinking and a relatively limited set of data-informed planning successes" but I must have missed the successful example.) So the entire article describes what they think should be done, without any successful examples to show that anything useful can be done in the first place.
wjnc 4y ago
Obviously I’m 20 years past university and nearer being the paying parent, but why oh why should university “capture the value of data”. They should offer affordable education (if you believe in the value of tertiary education) and affordable signaling of abilities (if you don’t). A recent post by John Cochrane [1] somewhat pointed out the absurdity in US education, Stanford specifically. Stanford itself mentions 15750 non teaching staff members on 2288 professors. My kids K12 (EU) runs on about 6 administrators (excluding cleaning, but including all other staff) for 22 teaching staff, including upstream shared services in the city. In a high school the ratio is probably even better. 15750 administrators wanting to capture the value of data is just plain silly. (I’m going overboard here, but it’s about an order of magnitude of difference! I know of 50% differences in FTE/value in the market I work in, but x10…) [1] https://johnhcochrane.blogspot.com/?m=1 https://johnhcochrane.blogspot.com/?m=1
- Retric 4y ago
  Public schools depend on a lot of administrative support outside the physical school its self. Buss drivers for example aren’t managed at the individual school level because they transport students to multiple different schools. This extends through a huge range of administrative functions for everything from calling snow days to collecting taxes to pay for the school etc.
- ordersofmag 4y ago
  A big chunk of that 15,000 are professionals engaged in research activities (mostly funded by grants and other external funds) e.g. the 1600 folks involved in running the Stanford Linear Accelerator, and activities like fundraising and finances. They mostly are NOT folks 'administering' the faculty in any way. So the comparison with your kid's school is not meaningful. Not to say there are perhaps more administrators than might be ideal. But these numbers aren't helpful in understanding whether that's true. Also, it's worth noting that Stanford is in fact free for folks under the median household income in the U.S. (roughly speaking) which seems pretty 'affordable'. Of course the economics of one of these large-endowment, research intensive institutions is pretty much unrelated to their teaching function. But that just highlights the weakness of using gross, whole-institution numbers of people/dollars for any sort of comparison. Big universities serve lots of ends (not just teaching undergrads) and so the teasing apart the economic picture (and whether it's efficient at meeting it's many goals) is complicated.
civilized 4y ago
As a data scientist, I think most data is useless, but there is an addictive, video game-like quality to throwing lifeless spreadsheets into a machine and having colorful visualizations come out. It's kind of like a very boring video game for adults that makes them feel like they're working, when they're actually just enjoying colorful abstract shapes and colors. To be honest, this is probably a sizable piece of why I'm in this line of work.
- foreverCarlos 4y ago
  Agreed! It all probably started with big tech promoting the “data-driven decision” paradigm. Of course, in many cases this approach is effective, but it’s not a panacea and has its limits. It’s tempting to interpret availability of any data as an amazing untapped resource, but in many cases analyzing it is just a massive waste of resources and could be more effectively replaced with traditional tools (surveys and such).
- nonrandomstring 4y ago
  > It's kind of like a very boring video game for adults that makes them feel like they're working, Absolutely love it. Let's distinguish a few things though. "Data science" seems like a pretty weird name. I mean, it's just "Science" right. Of course there's statistics, mathematics, signal processing, systems analysis, machine learning... all the good things that you and I are into. But how does this get huddled uncomfortably beneath the umbrella "Data science"? I think the answer is found by asking about the ends of data science, the old Cui Bono? There's the raw entertainment value you mention. It's cool to have knowledge and visualise it. Sensors, transducers, processing is fun. Then there's legibility. That is political and is about control. What most scientists are doing with data is either hypothesis testing or combing for causal relations to then abductively feed back into hypothesis generation. What most business people are trying to do is optimise, and adjust constraints and parameters. It's modelling for the most-part. It's ancient and goes back to linear analysis and regression from before the last century. Security people are looking for stress signifiers, suspicious patterns with various triggers, selectors and tripwires. Financial people want fortune telling. They want the models to extrapolate into beautiful hockey sticks. Within any organisation we may need to do one, a few, many or none at all of the above. The problem then is that "Valuable data" is such a broad, open prospect it seduces gushing, credulous administrators into valuing the process, and the tools, but not the ends.
bee_rider 4y ago
It seems like a somewhat weird mismatch — the described benefits are things like curriculum improvements and hiring better instructors, which I think most people (including most professors, I think, despite the fact that they’d be on the receiving end of this) would be in favor of. But including security cameras in the list of tools seems… odd? What would they tell you? Classroom attendance I guess. But all that tells you is that the instructor is either so bad that nobody bothers showing up, or so good that their slides, recordings, etc are enough for the students to skip class sometimes. (And if you measure the number of students in the classroom, attendance will just become mandatory, this is dumb). In general universities should be pretty inefficient I think. They are places where young people go to learn and try things out. Including student employees. If a university has no waste, that indicates that it hasn’t given enough possibly-unqualified undergrads enough resources to accidentally misuse.
- yucky 4y ago
  > In general universities should be pretty inefficient I think. They are places where young people go to learn and try things out. Including student employees. If a university has no waste, that indicates that it hasn’t given enough possibly-unqualified undergrads enough resources to accidentally misuse. The problem is all the extra money (yes, pretty much all of it) has gone to building up the administrations, not into resources that actually help students. And the students certainly haven't turned out smarter as a result. Universities have turned into a jobs program for people looking to work in education administration.
gchallen 4y ago
Working in a top-tier computer science department, I find our ability to answer basic questions about the health of our degree program fairly troubling. I think non-academics may be surprised by how much we don't know, and how little useful and continuous data analysis is taking place. For example: What is our retention rate? Meaning, what percentage of students who start our degree programs complete it. A fairly standard and important indicator of program health. Next, break this down by various cohorts: What is our retention rate among women? And so on. Heck, frequently we can't even answer questions about the current gender ratio within our program—and this is something that has been a focus of our diversity efforts recently. I've had people say with a straight face that we _cannot_ calculate retention because we don't know when students leave our program. But of course someone knows this! And I've been able to produce rough estimates even given the limited data that I have access to. But a lot of educational data is fairly siloed, and frequently the people assigned to perform these tasks don't have much training and tend to give up quickly. I suspect that many departments just don't have anyone assigned to do even basic educational data analysis on a regular basis, and with access to enough data to run interesting reports. My department is in the process of creating a faculty leadership role around academic data analytics, but my sense is that this will be a very unusual position. (And don't worry—it'll be filled by a faculty member, and not a new administrator.) And don't even get me started about student evaluations of teaching. Yes, we give a survey at the end of every semester and ask students whether they liked a particular course and professor. No, those answers have very little to do with how much they actually learned. Yes, we could measure learning in other better ways—success in downstream courses, for example. No, people don't tend to do that. There's a lot of room for improvement here, just working with the data we already have. No need for additional "telemetric signals".
- nobrains 4y ago
  OK, then what? Then you find out why retention is low. Then you brainstorm ideas to increase retention. Then you attempt to apply those ideas. It is at this point that the person responsible for applying those ideas says "been there done that". Point is, most data dashboards are non actionable. The challenge is to create a good actionable dashboard (i.e. if values cross a certain threshold, then the user should take some action on it). Once you create an excellent actionable dashboard, you realize it doesn't need a dashboard. It can be a notification. So, while the data is important, the questions around it might just lead to the same work that was being done anyway.
MengerSponge 4y ago
Oh, so we need a Vice-provost of Data. That's what is keeping universities, otherwise homogenous and well structured, from aligning on data infrastructure and strategy.
eruci 4y ago
Then give the data away for free to those who might do something useful with it.
colechristensen 4y ago
Honestly, down with data science. Down with university administration. Stop running dragnet data collection on unwilling participants and stop using it to try to “optimize” your interactions with them. This data shouldn’t exist, it’s value shouldn’t be exploited.
lifeisstillgood 4y ago
Because it does not "belong" to them. It belongs to all science and all society, whom will through creative destruction find ways to extract the value for everyone. The same applies to corporations, landholders, monarchs and parliaments and governments. New year is bringing out my fundamentals :-)
hammock 4y ago
The abstract suggests higher ed administrators collect data from “research, administrative operations and other sources” but there is one source not mentioned: donors. I am absolutely sure that donor data is organized, systematized, useful and generates positive ROI. It’s surprising that this study would not mention donor data (unless they had a predetermined “struggle with data” point of view in mind)
wolverine876 4y ago
> The study’s authors contend that universities have been slower than organizations in other economic sectors to create senior-level positions focused on data quality, strategy, governance and privacy matters. Universities are not an economic sector; they aren't profit-making enterprises. They are knowledge- and education-making enterprises. Do they need more employees who aren't creating knowledge and educating? My understanding is that universities have been greatly expanding such non-core functions.
- fullsend 4y ago
  That’s exactly what universities are these days. They sell a signal that says “this worker can do the bare minimum you need” called a Degree. We make the worker get certified themselves, but this is the purpose. To act like modern Universities do anything else today is disingenuous in my opinion. It’s like saying people who work in finance are there to allocate capital efficiently. No, they’re there to make money. That’s it.
  - wolverine876 4y ago
    It's a popular meme but IMHO does not reflect university-level analysis. That exists, but reductionist arguments merely take the worst of someone or something and blow that up to 100%. For example, everyone feels self-interest, but the reductionist claim it that it's all we are. A simple look at the evidence shows that it's manifestly untrue; people are much more than self-interested. Similarly universities do and are much more. I know university students and they are learning a lot - I'm very impressed with the thought and creative effort put into the cirriculum. The reductionists really hurt themselves and people who listen to them. They wall themselves off from all the good in the world, including education.
    - fullsend 4y ago
      True, the reductionism is no good. I guess to me I’m tired of discourse that leans towards “universities are places where we make you a complete human”. The real state of things is more what I describe, with a dash of making you deeper. Not vice versa.
etempleton 4y ago
There are a couple of challenges at play: 1. Systems in higher Ed often do not talk and departments tend to be siloed and have drastically different goals and objectives. 2. There are a lot of privacy concerns as most of the data is students data. In addition to this, I question what the end goal of the data analysis is? What is the value for a not-for-profit? The goal is going to be based on the department as a college doesn’t have the same profit motive. The individual departments typically don't have the resources to do deep data analysis.
paultopia 4y ago
There are so so so so many reasons for this. Research data: hard to centralize because subject to impossible conflicting demands from funders and human subjects IRBs, and in the hands of ferociously independent faculty. Students: privacy laws and worse than that over-cautious university counsel and administrators who are afraid of running afoul of those laws. Administrative data: stuck in a bunch of individual bureaucratic buckets that don't talk to one another because universities are badly managed in general, also in the clutches of horrific enterprise software platforms made by the worst companies in the world (like oracle) and administered by IT people who aren't paid market rates. University data projects tend to succeed if and only if they're turned over to librarians, who are typically the only people on campus who have any clue how to do such a thing.
jrm4 4y ago
Data is not information. Lots of data is worthless if you don't know what you're looking for. Perhaps we could actually do something about this if we focused strongly on falsification, but right now there's just going to be a whole lot of Wittgenstein's ruler going on; what people want to look at or see will come first with such an abundance, and there's going to need to be some kind of real filter to make it useful.
alexb_ 4y ago
If it's bad in universities, how much of this can we expect to be true for the entire "big data" economy we've built a gigantic part of the internet off of?

University data is getting outdated very quickly. Large companies have taken over the role of data curators. I personally think this is wrong. Big tech companies should build hardware, while government agencies should use that hardware to curate our data. Like how it was near the beginning of the internet. Companies should be as far away from our data as possible.

robertlagrant 4y ago
I think the exact opposite: government should know as little as is required to provide its services, as should any company. If you imagine a company can do damage with your data, wait til a bad government has it.
- deleted 4y ago
  [deleted]

jimlongton 4y ago

I can't access the original study however taking data, explicitly naming security cameras for example, to "use it or merge it with data from external parties such as publishers or public or private sector organizations" will surely not seriously degrade student and staff privacy, right? The rush to "exploit" data reminds me of the dot com hype. It's one thing to use available data to make more informed decisions about things from course content to building occupancy. It's quite another to rush towards total surveillance because of a Fear of Missing Out of "exploitable data".

McSinyx 4y ago
> degrade student and staff privacy That's the primary value to be captured right there! Their privacy is highly valued, monetarily: > authors contend that universities have been slower than organizations in other _economic_ sectors Education in the US is just pure business.
- a9h74j 4y ago
  Make of it what you will, I just saw on the project censored website that protecting student data is seen by some as a major priority. The article seems more PII and cybersecurity focussed, rather than recognizing the threat of intentional monetization. https://www.projectcensored.org/ferpa-and-higher-ed-should-prioritize-the-safety-of-students-private-data/ https://www.projectcensored.org/ferpa-and-higher-ed-should-p...
  - nerdponx 4y ago
    This is about digital security, i.e. protecting data from unauthorized access and use. It says nothing about protecting students themselves from surveillance and abuse by their own institutions.

tomrod 4y ago

This is the type of article that makes me wonder: "therefore, what?" Yes, absolutely, universities are awash in data they do little to anything with outside of hyperfocused laboratories. I wonder if having central data management groups would be an enabler or a bottleneck, however.

Dwolb 4y ago
Therefore they can’t optimize anything. They have 0 visibility into any of the operations and therefore can’t derive any insights or install controls. They have no idea if: Some professors more effective teachers than others. Which days students buy more food. Whether taking a given prerequisite leads to better mastery of advanced material. If additional spend is correlated with better student outcomes. For people in tech businesses, it’s such a shocking thing to read it’s hard to fathom how any of the leadership at universities find this situation acceptable.
- jimlongton 4y ago
  I didn't go to a prestigious US university but even mine had all this data and used it. In tech we like to think we are the only ones to "get" data. We are not. Teaching outcomes are evident from exam results, grants won, papers published etc and are routinely used to determine promotions, staffing and more. Food/footfall etc are monitored and predicted by the private catering companies on campus. Budget reviews are comprehensive and require yearly and greater analysis of university performance on a per department basis. All that doesn't even include competitive processes like ranking tables for universities, applications for grants, participation in international research groups etc etc.
- miguelazo 4y ago
  > They have no idea if: Some professors more effective teachers than others. What are you even talking about? Literally every course at most universities end with an evaluation. Obviously, there are other problematic ways to measure effectiveness (publications). How much that data gets used is another question and varies widely by institution. “People in tech businesses” love to exaggerate the quality of the data they have and its usefulness, hence the Subprime Attention Crisis that is rapidly eroding their cash on hand as the Fed’s cheap money and QE dries up.

ghiculescu 4y ago

This article feels like it took hundreds of words to say nothing at all. The funniest part was the example of security cameras. What big breakthroughs are universities hoping to achieve with this data?

mistrial9 4y ago
some industry players want to normalize facial recognition and location tracking
- jimlongton 4y ago
  I had to laugh at one of the authors being Christine L. Borgman. I'm sure all our data will be treated respectfully when fully integrated into their unaccountable profit-driven total surveillance system.
nonrandomstring 4y ago
They're being pushed very hard from the supply side while lacking much push-back internally due to dwindling expertise and leadership. This 4 parter I wrote for Techrights [1] and the Times article that seeded it [2] go into detail. [1] http://techrights.org/2022/12/28/andy-farnell-on-british-universities/ http://techrights.org/2022/12/28/andy-farnell-on-british-uni... [2] https://www.timeshighereducation.com/campus/eliminating-harmful-digital-technologies-universities-guide https://www.timeshighereducation.com/campus/eliminating-harm...
deleted 4y ago
[deleted]
foreverCarlos 4y ago
Security cameras in universities seem to be accomplishing… nothing. I am getting very frequent emails (2-3 per week) from one of the top US universities with blurry unusable screenshots of perpetrators stealing items, breaking in, assaulting students, etc. Makes me think - should they not invest into better security instead?

nonrandomstring 4y ago

There's a troubling tone behind this article. It's a prophecy that demands to be fulfilled. Starting with the premise "Data is useful", it proceeds to pick at all the ways we've failed to make it useful... and we're damned well going to make it useful if it kills us! Maybe, just maybe (for those that dabble in the sceptical, explanatory game we call science) it might be that "bare data" has little use within certain contexts... say those that by definition are on the cutting edge of knowledge and reality, best steered by imaginative vision of leading experts. Data driven market cybernetics may be very useful in some industries, such as ones with physical logistics, complex supply chains, rapidly shifting supply sources and demand sinks. But the primacy of "data uber alles" should not be one-size-fits-all.

lukev 4y ago
There's a cost/benefit curve here. Having data can undeniably be useful. The question is, how much ROI does the insights in the data provide, relative to the cost to build the organizational infrastructure capable of extracting these insights? And there's a bootstrapping problem, because often it's hard to calculate the potential ROI without actually going through the work of building the infrastructure to process the data.
- nonrandomstring 4y ago
  Indeed, absolutely true. And furthermore, sometimes the value of data, and of possible other sources, doesn't emerge until you start analysis and then deductively see new collection opportunities. It's a to-and-fro process. > how much ROI does the insights in the data provide, relative to the cost to build the organizational infrastructure Having seen the workings of academia for a long time now I'd argue very little. But I think the problem is more subtle. A Heisenbergian problem. When data collection involves people you have a triple problem of intrusion, distraction, and ossification. the act of trying to create a "data driven culture" in some contexts simply kills what's there. The cost of this, in addition to the bootstrapping costs of sensors and analytics are paralysing. Academia, by its very nature, if properly functioning, demands to be dynamic. If it's good, it will not stand still long enough for you to look at it. And if you measure it "too hard" it will evaporate as all the good people who resent ossification leave.

lukev 4y ago

I have no doubt the problem is worse in academia, but to be quite honest nobody is really doing a good job at curating, governing and unifying data across large organizations (at least without extreme expense.) Just look at the treadmill of architectures designed to "solve" this problem: integrated databases to "data warehouses" to "linked data" to "data lakes" to "data fabric" to "data mesh." Organizations that succeed do so because they have the resources to prioritize data management, and do so at significant team size and expense. But any organization that doesn't (or can't) view data curation as first class, top-line budget isn't in a position to capture even a small part of the theoretical value of their data. Consider the "data mesh" architecture, the latest fad in this space. It basically assumes that not only do you have a dedicated data curation team, each data source also has it's own dedicated team of curators capable of productizing data sources. That kind of thing is simply out of reach for most organizations.

le-mark 4y ago
I lived through an acquisition in the teens that I feel was an illustrative story. An old financial services provider embarked on a massive, whole organization modernization effort. The new CEO went all in and actually made the necessary investment in staff and headcount, which was remarkable in itself. A big part of the effort involved providing customers with big data solutions and all the implied benefits that would entail. They jumped aboard the hype train with gusto. Each quarter progress was reported and margins improved with a clear upward trend. From the trenches it was clear that even with the investment made, it was still not enough to reach any of the goals. After a year of “modernization” the company was acquired, the buyer literally salivating over the coming profit margin increase and “synergies” in selling into related markets. After another year post acquisition it became clear that the benefits were simply out of reach, and the profits did not arrive. The acquiring company was left with the low margin business it had always been. Layoffs ensued as the buyer sought to salvage something from the transaction. Turned out the entire modernization and big data effort had been an elaborate bait and switch committed by the board and CEO of the acquired company, masterfully executed.
- WrtCdEvrydy 4y ago

Eumenes 4y ago

We need more ad-tech startups for the university/higher education industry NOW

a9h74j 4y ago
One model: YT for persons without an adblocker. Or: This lecture brought to you by Pepsi.
musicale 4y ago
> We need more ad-tech startups for the university/higher education industry NOW I had the same thought - intrusive analytics and a focus on exploiting and monetizing client data seem to have damaged the web, made gaming less fun, and made everything user-hostile and creepy. Do we really want universities to go down that path?

pcrh 4y ago

Universities already waste vast sums of money of vice-provosts, middle management, administration, etc. There's no need to encourage them to start treating their mission as one of data mining in order to "capture" more value from the students.

foreverCarlos 4y ago
While I agree with your general skepticism, I also believe there are ways these data could benefit the students. Extracting value from operational data could mean modernizing curriculum, offering more office hours, etc.
- civilized 4y ago
  I think most of the value of "data" is captured in empowering individual contributors to observe their working conditions and the impact of their actions and adapt their strategy in response. The power of the central office spreadsheet wielders is very secondary. (Which is of course not to say that spreadsheets are not valuable, as any teacher knows.)
  - foreverCarlos 4y ago
    In an ideal world - yes. But seeing how overworked professors and other staff members already are in top universities, I doubt this is top of mind for them. There are courses with hundreds of enrolled students, weekly homework, practical projects, etc. IMO somewhat centralized DS tools are more suited to handle this load than an individual contributor (a professor in this context).
blthree 4y ago
You have a point about the amount of money spent on admin at Universities, but I think you (and most commenters here) don't really realize how bad the data situation is at a lot of major universities. If we want to streamline admin overhead at public universities, there's an incredible amount of time and money wasted due to extremely inefficient use/access to data. I work as a BI dev at a large Big 10 school (50k students/35k faculty and staff). For all those students and staff, there are a grand total of 3 data engineers, who are responsible for all our central data warehouses, including DBAing our oracle and redshift DBs. Just to get a new (untransformed) table added to the data warehouse from a source takes 6+ months, and that's only if it's from a source with an existing integration. On top of that, there are at least 50 people I know of whose job is basically to produce one or two reports manually in excel every week, and this isn't even considering people in finance or accounting. I'm talking about simple things like how many active research grants do we have, or how many students have enrolled in certain courses. These are reports (and entire fte positions) that could easily be automated with a single SQL query. Speaking of SQL, outside of the data engineering team, there are only 4 or 5 people who know any SQL out of the 100 I know of in reporting/BI positions. The "advanced" data teams are using MS access as an ETL tool to pull together data for tableau reports. However, there are a lot of institutional issues that make fixing those problems difficult. For one, while we have a central IT dept, we also have about 10 individual college-level IT teams, which means that data isn't just in different databases, but on a whole separate network. For example, if I want to create a report on student faculty ratio, I need to connect to VPN 1 to export faculty data from redshift, then switch to VPN 2 to export student data from Oracle, then switch to VPN 3 so that I can upload both datasets to our depts SQL server. After all that I can finally write a SQL query to get a student faculty ratio. Oh and when we need to update that ratio in a month, I'll have to go thru the whole manual extract/load process again. Forget automating that, since the network teams have no incentive to allow any tunnelling or bridging from one network to another. I could rant about this all day, but I think it's fair to say that there are still a ton of low-hanging fruit inefficiency-wise at universities. If we could get universities to value their data more highly, maybe that wod have the additional effect of solving some of these problems and even be a net money saver.

zitterbewegung 4y ago

I was in a social network research laboratory. Some of the best people at the lab went into industry while the others did do post docs. The professor that worked at the lab also went to Microsoft Research. Honestly if you were to go the route of being a professor or from MANGA what would you choose? Also, I think there are laws and also rules within Universities that would preclude the study to capture value and even then you would have to hire data analysts that would be external to being a professor so I think it would be hard to compete with MANGA in that regard also.

carbocation 4y ago

Oh good - an opportunity to use "begging the question" for its original meaning. I don't see a single example of "success" in the article. (They said "We unexpectedly found a pervasive void of infrastructure thinking and a relatively limited set of data-informed planning successes" but I must have missed the successful example.) So the entire article describes what they think should be done, without any successful examples to show that anything useful can be done in the first place.

wjnc 4y ago

Obviously I’m 20 years past university and nearer being the paying parent, but why oh why should university “capture the value of data”. They should offer affordable education (if you believe in the value of tertiary education) and affordable signaling of abilities (if you don’t). A recent post by John Cochrane [1] somewhat pointed out the absurdity in US education, Stanford specifically. Stanford itself mentions 15750 non teaching staff members on 2288 professors. My kids K12 (EU) runs on about 6 administrators (excluding cleaning, but including all other staff) for 22 teaching staff, including upstream shared services in the city. In a high school the ratio is probably even better. 15750 administrators wanting to capture the value of data is just plain silly. (I’m going overboard here, but it’s about an order of magnitude of difference! I know of 50% differences in FTE/value in the market I work in, but x10…) [1] https://johnhcochrane.blogspot.com/?m=1 https://johnhcochrane.blogspot.com/?m=1

Retric 4y ago
Public schools depend on a lot of administrative support outside the physical school its self. Buss drivers for example aren’t managed at the individual school level because they transport students to multiple different schools. This extends through a huge range of administrative functions for everything from calling snow days to collecting taxes to pay for the school etc.
ordersofmag 4y ago
A big chunk of that 15,000 are professionals engaged in research activities (mostly funded by grants and other external funds) e.g. the 1600 folks involved in running the Stanford Linear Accelerator, and activities like fundraising and finances. They mostly are NOT folks 'administering' the faculty in any way. So the comparison with your kid's school is not meaningful. Not to say there are perhaps more administrators than might be ideal. But these numbers aren't helpful in understanding whether that's true. Also, it's worth noting that Stanford is in fact free for folks under the median household income in the U.S. (roughly speaking) which seems pretty 'affordable'. Of course the economics of one of these large-endowment, research intensive institutions is pretty much unrelated to their teaching function. But that just highlights the weakness of using gross, whole-institution numbers of people/dollars for any sort of comparison. Big universities serve lots of ends (not just teaching undergrads) and so the teasing apart the economic picture (and whether it's efficient at meeting it's many goals) is complicated.

civilized 4y ago

As a data scientist, I think most data is useless, but there is an addictive, video game-like quality to throwing lifeless spreadsheets into a machine and having colorful visualizations come out. It's kind of like a very boring video game for adults that makes them feel like they're working, when they're actually just enjoying colorful abstract shapes and colors. To be honest, this is probably a sizable piece of why I'm in this line of work.

foreverCarlos 4y ago
Agreed! It all probably started with big tech promoting the “data-driven decision” paradigm. Of course, in many cases this approach is effective, but it’s not a panacea and has its limits. It’s tempting to interpret availability of any data as an amazing untapped resource, but in many cases analyzing it is just a massive waste of resources and could be more effectively replaced with traditional tools (surveys and such).
nonrandomstring 4y ago
> It's kind of like a very boring video game for adults that makes them feel like they're working, Absolutely love it. Let's distinguish a few things though. "Data science" seems like a pretty weird name. I mean, it's just "Science" right. Of course there's statistics, mathematics, signal processing, systems analysis, machine learning... all the good things that you and I are into. But how does this get huddled uncomfortably beneath the umbrella "Data science"? I think the answer is found by asking about the ends of data science, the old Cui Bono? There's the raw entertainment value you mention. It's cool to have knowledge and visualise it. Sensors, transducers, processing is fun. Then there's legibility. That is political and is about control. What most scientists are doing with data is either hypothesis testing or combing for causal relations to then abductively feed back into hypothesis generation. What most business people are trying to do is optimise, and adjust constraints and parameters. It's modelling for the most-part. It's ancient and goes back to linear analysis and regression from before the last century. Security people are looking for stress signifiers, suspicious patterns with various triggers, selectors and tripwires. Financial people want fortune telling. They want the models to extrapolate into beautiful hockey sticks. Within any organisation we may need to do one, a few, many or none at all of the above. The problem then is that "Valuable data" is such a broad, open prospect it seduces gushing, credulous administrators into valuing the process, and the tools, but not the ends.

bee_rider 4y ago

It seems like a somewhat weird mismatch — the described benefits are things like curriculum improvements and hiring better instructors, which I think most people (including most professors, I think, despite the fact that they’d be on the receiving end of this) would be in favor of. But including security cameras in the list of tools seems… odd? What would they tell you? Classroom attendance I guess. But all that tells you is that the instructor is either so bad that nobody bothers showing up, or so good that their slides, recordings, etc are enough for the students to skip class sometimes. (And if you measure the number of students in the classroom, attendance will just become mandatory, this is dumb). In general universities should be pretty inefficient I think. They are places where young people go to learn and try things out. Including student employees. If a university has no waste, that indicates that it hasn’t given enough possibly-unqualified undergrads enough resources to accidentally misuse.

yucky 4y ago
> In general universities should be pretty inefficient I think. They are places where young people go to learn and try things out. Including student employees. If a university has no waste, that indicates that it hasn’t given enough possibly-unqualified undergrads enough resources to accidentally misuse. The problem is all the extra money (yes, pretty much all of it) has gone to building up the administrations, not into resources that actually help students. And the students certainly haven't turned out smarter as a result. Universities have turned into a jobs program for people looking to work in education administration.

gchallen 4y ago

Working in a top-tier computer science department, I find our ability to answer basic questions about the health of our degree program fairly troubling. I think non-academics may be surprised by how much we don't know, and how little useful and continuous data analysis is taking place. For example: What is our retention rate? Meaning, what percentage of students who start our degree programs complete it. A fairly standard and important indicator of program health. Next, break this down by various cohorts: What is our retention rate among women? And so on. Heck, frequently we can't even answer questions about the current gender ratio within our program—and this is something that has been a focus of our diversity efforts recently. I've had people say with a straight face that we _cannot_ calculate retention because we don't know when students leave our program. But of course someone knows this! And I've been able to produce rough estimates even given the limited data that I have access to. But a lot of educational data is fairly siloed, and frequently the people assigned to perform these tasks don't have much training and tend to give up quickly. I suspect that many departments just don't have anyone assigned to do even basic educational data analysis on a regular basis, and with access to enough data to run interesting reports. My department is in the process of creating a faculty leadership role around academic data analytics, but my sense is that this will be a very unusual position. (And don't worry—it'll be filled by a faculty member, and not a new administrator.) And don't even get me started about student evaluations of teaching. Yes, we give a survey at the end of every semester and ask students whether they liked a particular course and professor. No, those answers have very little to do with how much they actually learned. Yes, we could measure learning in other better ways—success in downstream courses, for example. No, people don't tend to do that. There's a lot of room for improvement here, just working with the data we already have. No need for additional "telemetric signals".

nobrains 4y ago
OK, then what? Then you find out why retention is low. Then you brainstorm ideas to increase retention. Then you attempt to apply those ideas. It is at this point that the person responsible for applying those ideas says "been there done that". Point is, most data dashboards are non actionable. The challenge is to create a good actionable dashboard (i.e. if values cross a certain threshold, then the user should take some action on it). Once you create an excellent actionable dashboard, you realize it doesn't need a dashboard. It can be a notification. So, while the data is important, the questions around it might just lead to the same work that was being done anyway.

MengerSponge 4y ago

Oh, so we need a Vice-provost of Data. That's what is keeping universities, otherwise homogenous and well structured, from aligning on data infrastructure and strategy.

eruci 4y ago

Then give the data away for free to those who might do something useful with it.

colechristensen 4y ago

Honestly, down with data science. Down with university administration. Stop running dragnet data collection on unwilling participants and stop using it to try to “optimize” your interactions with them. This data shouldn’t exist, it’s value shouldn’t be exploited.

lifeisstillgood 4y ago

Because it does not "belong" to them. It belongs to all science and all society, whom will through creative destruction find ways to extract the value for everyone. The same applies to corporations, landholders, monarchs and parliaments and governments. New year is bringing out my fundamentals :-)

hammock 4y ago

The abstract suggests higher ed administrators collect data from “research, administrative operations and other sources” but there is one source not mentioned: donors. I am absolutely sure that donor data is organized, systematized, useful and generates positive ROI. It’s surprising that this study would not mention donor data (unless they had a predetermined “struggle with data” point of view in mind)

wolverine876 4y ago

> The study’s authors contend that universities have been slower than organizations in other economic sectors to create senior-level positions focused on data quality, strategy, governance and privacy matters. Universities are not an economic sector; they aren't profit-making enterprises. They are knowledge- and education-making enterprises. Do they need more employees who aren't creating knowledge and educating? My understanding is that universities have been greatly expanding such non-core functions.

fullsend 4y ago
That’s exactly what universities are these days. They sell a signal that says “this worker can do the bare minimum you need” called a Degree. We make the worker get certified themselves, but this is the purpose. To act like modern Universities do anything else today is disingenuous in my opinion. It’s like saying people who work in finance are there to allocate capital efficiently. No, they’re there to make money. That’s it.
- wolverine876 4y ago
  It's a popular meme but IMHO does not reflect university-level analysis. That exists, but reductionist arguments merely take the worst of someone or something and blow that up to 100%. For example, everyone feels self-interest, but the reductionist claim it that it's all we are. A simple look at the evidence shows that it's manifestly untrue; people are much more than self-interested. Similarly universities do and are much more. I know university students and they are learning a lot - I'm very impressed with the thought and creative effort put into the cirriculum. The reductionists really hurt themselves and people who listen to them. They wall themselves off from all the good in the world, including education.
  - fullsend 4y ago
    True, the reductionism is no good. I guess to me I’m tired of discourse that leans towards “universities are places where we make you a complete human”. The real state of things is more what I describe, with a dash of making you deeper. Not vice versa.

etempleton 4y ago

There are a couple of challenges at play: 1. Systems in higher Ed often do not talk and departments tend to be siloed and have drastically different goals and objectives. 2. There are a lot of privacy concerns as most of the data is students data. In addition to this, I question what the end goal of the data analysis is? What is the value for a not-for-profit? The goal is going to be based on the department as a college doesn’t have the same profit motive. The individual departments typically don't have the resources to do deep data analysis.

paultopia 4y ago

There are so so so so many reasons for this. Research data: hard to centralize because subject to impossible conflicting demands from funders and human subjects IRBs, and in the hands of ferociously independent faculty. Students: privacy laws and worse than that over-cautious university counsel and administrators who are afraid of running afoul of those laws. Administrative data: stuck in a bunch of individual bureaucratic buckets that don't talk to one another because universities are badly managed in general, also in the clutches of horrific enterprise software platforms made by the worst companies in the world (like oracle) and administered by IT people who aren't paid market rates. University data projects tend to succeed if and only if they're turned over to librarians, who are typically the only people on campus who have any clue how to do such a thing.

jrm4 4y ago

Data is not information. Lots of data is worthless if you don't know what you're looking for. Perhaps we could actually do something about this if we focused strongly on falsification, but right now there's just going to be a whole lot of Wittgenstein's ruler going on; what people want to look at or see will come first with such an abundance, and there's going to need to be some kind of real filter to make it useful.

alexb_ 4y ago

If it's bad in universities, how much of this can we expect to be true for the entire "big data" economy we've built a gigantic part of the internet off of?