18 ms·

AI Destroys Institutions

91 points by sean_the_geek 9mo ago

sean_the_geek 9mo ago
A thought provoking essay on impact of AI systems civic institutions.
chrisjj 9mo ago
Dupe of https://news.ycombinator.com/item?id=46622870 https://news.ycombinator.com/item?id=46622870
- embedding-shape 9mo ago
  Not really a dupe if no one really discussed it. And I'm glad, didn't see it that previous time :)
RobotToaster 9mo ago
[flagged]
- embedding-shape 9mo ago
  > > universities [...] empower individuals to take intellectual risks and challenge the status quo. What a misleading quote you made there. Why? Full quote: > Purpose-driven institutions built around transparency, cooperation, and accountability empower individuals to take intellectual risks and challenge the status quo. I'd agree with that universities more often than not don't fall into the group of "Purpose-driven institutions built around transparency, cooperation, and accountability", but that's a different thing. There are more institutions than universities.
  - RobotToaster 9mo ago
    The actual full quote specifically defines universities to be one of those institutions. >"Civic institutions—the rule of law, universities, and a free press—are the backbone of democratic life. They are the mechanisms through which complex societies encourage cooperation and stability, while also adapting to changing circumstances. The real superpower of institutions is their ability to evolve and adapt within a hierarchy of authority and a framework for roles and rules while maintaining legitimacy in the knowledge produced and the actions taken. Purpose-driven institutions built around transparency, cooperation, and accountability empower individuals to take intellectual risks and challenge the status quo."
    - embedding-shape 9mo ago
      > The actual full quote specifically defines universities to be one of those institutions. Exactly, one of, among others, and the paper is about a group of civic institutions, not specifically universities, that's why that quote is very misleading. The actual quote is talking about "Purpose-driven institutions built around transparency, cooperation, and accountability", I think that's pretty clear.
emsign 9mo ago
> The affordances of AI systems have the effect of eroding expertise, short-circuiting decision-making, and isolating people from each other. This affordability is HEAVILY subsidized by billionaires who want to destroy institutions for selfish and ideological reasons.
- DarkNova6 9mo ago
  This is literally corporate textbook 101. Subsidize your product, become market leader, cause lock-in and make your customers dependant. Every large enough corporate wants to become the new Oracle.
- ahartmetz 9mo ago
  Given how nobody properly understands LLMs, I doubt that they are intentionally designed like that. But the effect... yeah. I can see that happening. (By the way, are you confusing affordance, the UX concept, with affordability?)
  - esperent 9mo ago
    Nobody properly understands dog brains either and yet you can still train a dog to sit.
    - ahartmetz 9mo ago
      If you just met a dog for the first time, you can't :) - my guess is LLMs are somewhere in between. It would be cool to see what happens if somebody tried to make an LLM that somehow has ethical principles (instead of guardrails) and is much less eager to please.
      - chrisjj 9mo ago
        The stochastic parrot LLM is driven by nothing but eagerness to please. Fix that, and the parrot falls off its perch.
rfv6723 9mo ago
This dire warning against AI echoes the anxieties of a much earlier elite: the late-medieval clergy facing the invention of the printing press. For centuries, they held a privileged monopoly on knowledge, controlling its interpretation and dissemination. The printing press threatened to shatter that authority by democratizing access to information and empowering individuals. Similarly, today's critics, often from within the very institutions they defend, frame AI as a threat to "expertise" and "civic life" when in reality, they fear it as a threat to their own status as the sole arbiters of truth. Their resistance is less a principled defense of democracy and more a desperate attempt to protect a crumbling monopoly on knowledge.
- embedding-shape 9mo ago
  If what you say was true, why are people from not within those institutions also try to warn others about the potential downfall of "expertise" and "civic life"? Are they just misinformed? Paid by these "institutional defenders" or what is your hypothesis?
  - rfv6723 9mo ago
    The alarm isn't coming from outside the institutions; it's coming from a wider, more modern clergy. The new priestly class isn't defined by a specific building, but by a shared claim to the mastery of complex symbolic knowledge. The linguists who call AI a "stochastic parrot" are the perfect example. Their panic isn't for the public good; it's the existential terror of seeing a machine master language without needing their decades of grammatical theory. They are watching their entire intellectual paradigm—their very claim to authority—be rendered obsolete. This isn't a grassroots movement. It's an immune response from the cognitive elite, desperately trying to delegitimize a technology that threatens to trivialize their expertise. They aren't defending society; they're defending their status.
    - raincole 9mo ago
      > Their panic isn't for the public good; it's the existential terror of seeing a machine master language without needing their decades of grammatical theory. It's some wild claim. Every linguist worth their salt had known that you don't need grammatical theory to reach native level. Grammar being descriptive rather than prescriptive is the mainstream idea and had been long before LLM. If you actually ask them, I bet most linguists will say they are not even excellent English (or whichever language they studied the most) teachers. Plus, "stochastic parrot" was coined before ChatGPT. If linguists really felt that threatened by the time when people's concerns over AI was like "sure it can beat go master but how about league of legends?" you have to admit they did have some special insights, right?
paganel 9mo ago
> free press Stopped reading here, as these people still believe in that fairytale of theirs.
- tlb 9mo ago
  The American press isn't perfectly free, but you should see what a state-controlled press is like.
  - paganel 9mo ago
    Bezos, whatever the heck this is [1], you people are delusional. Later edit: For good measure, Zuckerberg, too [2] [1] https://x.com/AdameMedia/status/2011935282912731453 https://x.com/AdameMedia/status/2011935282912731453 [2] https://x.com/infolibnews/status/2011196769363697684 https://x.com/infolibnews/status/2011196769363697684
YouAreWRONGtoo 9mo ago
[dead]
deleted 9mo ago
[deleted]
terminalshort 9mo ago
This is nothing but speculation written by lawyers in the format of a scientific paper to feign legitimacy. Of course those $500 an hour nitpickers are terrified of AI because it threatens the exorbitant income of their cartel protected profession.
- __loam 9mo ago
  Enough people have gotten owned for using these things in court that I think the more likely response is laughing at the ignorance then feeling threatened.
  - terminalshort 9mo ago
    1. Get owned in court because you used an LLM that made a poor legal argument. 2. Get owned out of court because you couldn't afford the $100K (minimum) that you have to pay to the lawyer's cartel to even be able to make your argument in front of a judge. I'll take number 1. At least you have a fighting chance. And it's only going to get better. LLMs today are the worst they will ever be, whereas the lawyer's cartel rarely gets better and never cuts its prices.
    - pousada 9mo ago
      Does it cost 100k minimum in the US to get a lawyer? Or am I misunderstanding something?
      - terminalshort 9mo ago
        There is no "get a lawyer." You pay by the hour. And there is months to years of procedure before the judge even knows your lawsuit exists.
        pousada 9mo ago
        And the minimum to file a lawsuit comes out to 100k at standard rates ? Or was it just a random number?
contrarian1234 9mo ago
Just from reading the abstract, it feels like the authors didn't even attempt at trying to be objective. It hard to take what they're saying seriously when the language is so loaded and full of judgments. The kind of language you'd expect in an Op-Ed and not a research paper
- DannyBee 9mo ago
  I think you may be confused. This is not a research paper, it's an op-ed in a law journal. SSRN is where most draft law review/journal articles are published, which may be the source of confusion. For most other fields, it is a source of draft/published science papers, but for law, it's pretty much any kind of article that is going to show up in a law review/journal.
  - chilmers 9mo ago
    It is literally called “ Boston Univ. School of Law Research Paper No. 5870623”
    - deleted 9mo ago
      [deleted]
    - DannyBee 9mo ago
      It's also an submission to UC hastings law journal, as it also says right before that? The automated tagging with a BUSL ID is just how BUSL's system for papers of any sort works. For reference: I did my first year of law school at BUSL so i'm very familiar with how it all works there :) This is also very common elsewhere - everything that IBM used to release got tagged with a technical report number too, for example, whether it was or not. In any case - it is clearly a piece meant to be persuasive writing, rather than deep research. Law journals contain a mix of essentially op-eds and deeper research papers or factual expositories/kind of thing. They are mostly not like scientific journals. Though some exist that are basically all op-ed or zero op-ed. Compare something like: https://repository.uclawsf.edu/cgi/viewcontent.cgi?article=2351&context=hastings_law_journal https://repository.uclawsf.edu/cgi/viewcontent.cgi?article=2... Which is a piece in UC law journal meant as an informative piece cataloguing how california courts adjudicate false advertising law. It does not really take a position. with https://repository.uclawsf.edu/cgi/viewcontent.cgi?article=3020&context=hastings_law_journal https://repository.uclawsf.edu/cgi/viewcontent.cgi?article=3... Which is a piece in UC hastings law journal meant as, essentially an op ed, arguing that dog sniff tests are bullshit. I picked both of these at random from stuff in UC hastings law journal that had been cited by the Supreme Court of California. There are things that are even more factual/take zero positions, and things that are even more persuasive writing/less researchy, than either of these, but they are reasonable representatives, i think
charcircuit 9mo ago
None of these paper's arguments are AI specific. The IRS doesn't need AI to make mistakes and be unable to tell you why it did so. You can find stories of that happening to people already.
- toofy 9mo ago
  i think when most people bring up mistakes that these models make, much of their concern is that little can be done. when one of the juniors makes a mistake, i can talk to them about it and help them understand where they went wrong, if they continue to make mistakes we can change their position to something more suited for them. we can always let them go if they have too much hubris to learn. who do we hold to account when a model makes a mistake? we’re already beginning to see, after major fuckups, companies blackhole nullrouting accountability into “not our fault, don’t look at us, ai was wrong” the other thing is, if you have done a good job selecting your team, you’ll have people who understand their limits, who understand when to ask for help, who understand when they don’t know something. a major problem with current models is that it will always just guess or stretch toward random rather than halt. so yes, people will make mistakes, but at least you can count on being able to mitigate for those after.
  - chrisjj 9mo ago
    > who do we hold to account when a model makes a mistake? First we stop anthromorphising the program as capable of making a "mistake". We recognise it merely as machine providing incorrect output, so we see the only mistake was made by the human who chose to rely upon it. The courts so far agree. Judges are punishing the gulled lawyers rather than their faux-intelligent tools.
  - terminalshort 9mo ago
    Who was held to account when the IRS made a mistake and sent me a demand letter for over $100K of "unpaid taxes" I didn't owe? Who compensated me for the hours I spent on hold and the money I had to pay an accountant to deal with it?
qsera 9mo ago
We should be more worried what AI will due to the ability of an average human to think. Not that I think there is a lot of thinking going on now anyway, thanks to our beloved smartphones. But just think about a time when human ability to reason has atrophied globally. AI might even give us true Idiocracy!
- fennecbutt 9mo ago
  I mean we were seeing this even before AI. It's the same type of person. To slop is human. It's like for some reason we thought that like some good percentage of us aren't just tribal worker drones who fundamentally just want fats, sugars, salts, dopamine and seratonin. People actively vote against things like UBI, higher corporate taxes, making utilities public. People actively choose to believe misinformation because it suits their own personal tribal narratives.
- t0bia_s 9mo ago
  Similar rhetoric was allways there with new technologies. Calculators, radio, cameras, phones, computers, smartphones, social networks... Regulations do more harm than learning process from mistakes.
  - getnewmaterial 9mo ago
    Yes we have this before and everytime it was correct.
    - t0bia_s 9mo ago
      What was correct? That we became idiots with new tools?
      - qsera 9mo ago
        >That we became idiots with new tools? Yes, a little bit each time. But AI will finish the job. Because earlier it was writing skills, or attention span that was at stake. This time it is literally the ability to think.
juggle-anyhow 9mo ago
Who do institutions serve? To me AI democratises information. Allows access to information that would normally be gatekept. AI reduces barriers, and they don't like that because those barriers gave them authority.
magpi3 9mo ago
> Purpose-driven institutions built around transparency, cooperation, and accountability empower individuals to take intellectual risks and challenge the status quo. I am not sure if I am off-topic, but I am having a lot of trouble with this statement. Institutions are often opaque, and I have never belonged to an institution that empowered me to "take intellectual risks and challenge the status quo." Quite the contrary.
- popalchemist 9mo ago
  "purpose-driven" is the relevant qualifier here.
intended 9mo ago
I fear the title of this article is going drive most of the conversation. I haven’t read through the whole thing yet, but so far the parts of the argument I can pull out are about how Institutions actually work, as in a collection of humans. AI, as it currently stands, interacts with humans themselves in ways that hollow out the kind of behavior we want from institutions. “ Perhaps if human nature were a little less vulnerable to the siren’s call of shortcuts, then AI could achieve the potential its creators envisioned for it. But that is not the world we live in. Short-term political and financial incentives amplify the worst aspects of AI systems, including domination of human will, abrogation of accountability, delegation of responsibility, and obfuscation of knowledge and control” An analogy that I find increasingly useful is that of someone using a forklift to lift weights at gym. There is an observable tendency when using LLMs, to cede agency entirely to the machine.
- chrisjj 9mo ago
  > Perhaps if human nature were a little less vulnerable to the siren’s call of shortcuts, then AI could achieve the potential its creators envisioned for it. I can't see how, given that potential is 99% shorcuts.
layer8 9mo ago
I was amused at how they quote War Games.
incomingpain 9mo ago
>Civic institutions—the rule of law, universities, and a free press—are the backbone of democratic life. They are the mechanisms through which complex societies encourage cooperation and stability, while also adapting to changing circumstances. https://www.pewresearch.org/politics/2025/12/04/public-trust-in-government-1958-2025/ https://www.pewresearch.org/politics/2025/12/04/public-trust... In the 1960s, trust in institutions was around 70%. In 2025 it's about 17%. This is not something like we had 70% until 2023 and then AI dropped our trust suddenly. If anything, AI doesnt even register on the graph. So correlation here is practically non-existent. Gallup and Pew have the similar trends for journalists and universities. You dont get to blame AI for this. Interesting bump and question. How did clinton improve reputation and then bush destroy it? Or is that a false hump?
- terminalshort 9mo ago
  It's not a false hump, but why do you assume the president caused it just because it happened at the same time?
cornholio 9mo ago
I would summarize the central claim of the paper as: the widespread use of AI to mediate human interaction will rob people of agency, understanding and skill development, as well as destroying the social links necessary to maintain and improve institutions, while at the same time allowing powerful unaccountable actors (AI cabal) to interject into those relations and impose their institutional goals; by "institution" we mean a shared set of beneficial social rules, not merely an organization tasked with promoting them, "justice" vs. "US justice system". The authors then break down the mechanisms by which AI achieves these outcomes (that seem quite reductive and dated compared to the frontier, for example they take it as granted that AI cannot be creative, that it can only work prospectively and can't react to new situations and events etc.), as well as exemplifying those mechanism already at work in a few areas like journalism and academia.
- palmotea 9mo ago
  And I think that's about right. Despite the marketing, I think AI (especially if the hyped capabilities arrive) will be one of the most destructive technologies ever invented. It only looks good to blinkered and deluded technocrats.
- _DeadFred_ 9mo ago
  AI is by it's nature an entropy machine.
reaperducer 9mo ago
AI Destroys Institutions Working as intended. WONTFIX.
roxolotl 9mo ago
This articles claims are also interesting to me in terms of large scale software systems. They authors say: > They (AI systems) delegitimization knowledge, inhibit cognitive development, short-circuit long term thinking processes, and isolate humans by displacing or degrading long term human connection. This is a pretty good summary of the worry I’ve seen expressed about extensive use to build large pieces of software. Large pieces of software aren’t just the code that describes them. They exist in some sense in their authors as well. Effective projects seek to expand understanding of software systems across the organization so that relevant decisions can be made about their future. By relegating much of the decision making around structure of the software to AI you lose the systemic knowledge shared across the organization.
TYPE_FASTER 9mo ago
> The affordances of AI systems have the effect of eroding expertise Recently, it so happened that I spent an hour reverse engineering and documenting a piece of a system. A co-worker asked a LLM to do the same. It generated some really nice documentation. The difference is, I (as a team member) now have the understanding. Generating the documentation does not increase the understanding of the team.
ChrisArchitect 9mo ago
Related: How Generative AI is destroying society https://garymarcus.substack.com/p/how-generative-ai-is-destroying-society https://garymarcus.substack.com/p/how-generative-ai-is-destr... (https://news.ycombinator.com/item?id=46616713 https://news.ycombinator.com/item?id=46616713)
mapotofu 9mo ago
I think the argument makes a central mistake in putting too much trust into institutions. I don’t disagree with the conclusions, but the premise around blindly trusting institutions simply because they’ve been around for a long time lost me from taking most of their arguments seriously, despite opening the article thinking I would agree.
notanastronaut 9mo ago
"Abundance of books makes men less studious" - 15th-century Venetian editor, Hieronimo Squarciafico
tim333 9mo ago
The arguments that "AI Destroys Institutions" seem pretty iffy. I scrolled down to see what institutions had been destroyed it went on about Elon Musk's Doge destroying stuff but their closing say USAID I think had zero to do with AI. I'm in the UK and I don't think any institutions have been destroyed or even noticeably harmed by AI. In the US there is general chaos under Trump so it may be hard to differentiate.
i-blis 9mo ago
> Civic institutions—the rule of law, universities, and a free press—are the backbone of democratic life. I disagree. The backbone of democratic life are the rule of law and freedom of speech, which makes a big difference. The press has historically been a counter-power inquiring into privileges and breaches of the rule of the law and thus promoted freedom of speech but almost only inasmuch it served the interest of the emerging merchant bourgeoisie . And we are long past that. Universities never have been liberal forces: they backed the Church and refused paradigm shifts. They still are very conservative even though in a peculiar sense, as leftist conservatives.
MohskiBroskiAI 9mo ago
[flagged]

91 points by sean_the_geek 9mo ago

sean_the_geek 9mo ago

A thought provoking essay on impact of AI systems civic institutions.

chrisjj 9mo ago

Dupe of https://news.ycombinator.com/item?id=46622870 https://news.ycombinator.com/item?id=46622870

embedding-shape 9mo ago
Not really a dupe if no one really discussed it. And I'm glad, didn't see it that previous time :)

RobotToaster 9mo ago

[flagged]

embedding-shape 9mo ago
> > universities [...] empower individuals to take intellectual risks and challenge the status quo. What a misleading quote you made there. Why? Full quote: > Purpose-driven institutions built around transparency, cooperation, and accountability empower individuals to take intellectual risks and challenge the status quo. I'd agree with that universities more often than not don't fall into the group of "Purpose-driven institutions built around transparency, cooperation, and accountability", but that's a different thing. There are more institutions than universities.
- RobotToaster 9mo ago
  The actual full quote specifically defines universities to be one of those institutions. >"Civic institutions—the rule of law, universities, and a free press—are the backbone of democratic life. They are the mechanisms through which complex societies encourage cooperation and stability, while also adapting to changing circumstances. The real superpower of institutions is their ability to evolve and adapt within a hierarchy of authority and a framework for roles and rules while maintaining legitimacy in the knowledge produced and the actions taken. Purpose-driven institutions built around transparency, cooperation, and accountability empower individuals to take intellectual risks and challenge the status quo."
  - embedding-shape 9mo ago
    > The actual full quote specifically defines universities to be one of those institutions. Exactly, one of, among others, and the paper is about a group of civic institutions, not specifically universities, that's why that quote is very misleading. The actual quote is talking about "Purpose-driven institutions built around transparency, cooperation, and accountability", I think that's pretty clear.

emsign 9mo ago

> The affordances of AI systems have the effect of eroding expertise, short-circuiting decision-making, and isolating people from each other. This affordability is HEAVILY subsidized by billionaires who want to destroy institutions for selfish and ideological reasons.

DarkNova6 9mo ago
This is literally corporate textbook 101. Subsidize your product, become market leader, cause lock-in and make your customers dependant. Every large enough corporate wants to become the new Oracle.
ahartmetz 9mo ago
Given how nobody properly understands LLMs, I doubt that they are intentionally designed like that. But the effect... yeah. I can see that happening. (By the way, are you confusing affordance, the UX concept, with affordability?)
- esperent 9mo ago
  Nobody properly understands dog brains either and yet you can still train a dog to sit.
  - ahartmetz 9mo ago
    If you just met a dog for the first time, you can't :) - my guess is LLMs are somewhere in between. It would be cool to see what happens if somebody tried to make an LLM that somehow has ethical principles (instead of guardrails) and is much less eager to please.
    - chrisjj 9mo ago
      The stochastic parrot LLM is driven by nothing but eagerness to please. Fix that, and the parrot falls off its perch.

rfv6723 9mo ago

This dire warning against AI echoes the anxieties of a much earlier elite: the late-medieval clergy facing the invention of the printing press. For centuries, they held a privileged monopoly on knowledge, controlling its interpretation and dissemination. The printing press threatened to shatter that authority by democratizing access to information and empowering individuals. Similarly, today's critics, often from within the very institutions they defend, frame AI as a threat to "expertise" and "civic life" when in reality, they fear it as a threat to their own status as the sole arbiters of truth. Their resistance is less a principled defense of democracy and more a desperate attempt to protect a crumbling monopoly on knowledge.

embedding-shape 9mo ago
If what you say was true, why are people from not within those institutions also try to warn others about the potential downfall of "expertise" and "civic life"? Are they just misinformed? Paid by these "institutional defenders" or what is your hypothesis?
- rfv6723 9mo ago
  The alarm isn't coming from outside the institutions; it's coming from a wider, more modern clergy. The new priestly class isn't defined by a specific building, but by a shared claim to the mastery of complex symbolic knowledge. The linguists who call AI a "stochastic parrot" are the perfect example. Their panic isn't for the public good; it's the existential terror of seeing a machine master language without needing their decades of grammatical theory. They are watching their entire intellectual paradigm—their very claim to authority—be rendered obsolete. This isn't a grassroots movement. It's an immune response from the cognitive elite, desperately trying to delegitimize a technology that threatens to trivialize their expertise. They aren't defending society; they're defending their status.
  - raincole 9mo ago
    > Their panic isn't for the public good; it's the existential terror of seeing a machine master language without needing their decades of grammatical theory. It's some wild claim. Every linguist worth their salt had known that you don't need grammatical theory to reach native level. Grammar being descriptive rather than prescriptive is the mainstream idea and had been long before LLM. If you actually ask them, I bet most linguists will say they are not even excellent English (or whichever language they studied the most) teachers. Plus, "stochastic parrot" was coined before ChatGPT. If linguists really felt that threatened by the time when people's concerns over AI was like "sure it can beat go master but how about league of legends?" you have to admit they did have some special insights, right?

paganel 9mo ago

> free press Stopped reading here, as these people still believe in that fairytale of theirs.

tlb 9mo ago
The American press isn't perfectly free, but you should see what a state-controlled press is like.
- paganel 9mo ago
  Bezos, whatever the heck this is [1], you people are delusional. Later edit: For good measure, Zuckerberg, too [2] [1] https://x.com/AdameMedia/status/2011935282912731453 https://x.com/AdameMedia/status/2011935282912731453 [2] https://x.com/infolibnews/status/2011196769363697684 https://x.com/infolibnews/status/2011196769363697684

YouAreWRONGtoo 9mo ago

[dead]

deleted 9mo ago

[deleted]

terminalshort 9mo ago

This is nothing but speculation written by lawyers in the format of a scientific paper to feign legitimacy. Of course those $500 an hour nitpickers are terrified of AI because it threatens the exorbitant income of their cartel protected profession.

__loam 9mo ago
Enough people have gotten owned for using these things in court that I think the more likely response is laughing at the ignorance then feeling threatened.
- terminalshort 9mo ago
  1. Get owned in court because you used an LLM that made a poor legal argument. 2. Get owned out of court because you couldn't afford the $100K (minimum) that you have to pay to the lawyer's cartel to even be able to make your argument in front of a judge. I'll take number 1. At least you have a fighting chance. And it's only going to get better. LLMs today are the worst they will ever be, whereas the lawyer's cartel rarely gets better and never cuts its prices.
  - pousada 9mo ago
    Does it cost 100k minimum in the US to get a lawyer? Or am I misunderstanding something?
    - terminalshort 9mo ago
      There is no "get a lawyer." You pay by the hour. And there is months to years of procedure before the judge even knows your lawsuit exists.
      - pousada 9mo ago
        And the minimum to file a lawsuit comes out to 100k at standard rates ? Or was it just a random number?

contrarian1234 9mo ago

Just from reading the abstract, it feels like the authors didn't even attempt at trying to be objective. It hard to take what they're saying seriously when the language is so loaded and full of judgments. The kind of language you'd expect in an Op-Ed and not a research paper

DannyBee 9mo ago
I think you may be confused. This is not a research paper, it's an op-ed in a law journal. SSRN is where most draft law review/journal articles are published, which may be the source of confusion. For most other fields, it is a source of draft/published science papers, but for law, it's pretty much any kind of article that is going to show up in a law review/journal.
- chilmers 9mo ago
  It is literally called “ Boston Univ. School of Law Research Paper No. 5870623”
  - deleted 9mo ago
    [deleted]
  - DannyBee 9mo ago
    It's also an submission to UC hastings law journal, as it also says right before that? The automated tagging with a BUSL ID is just how BUSL's system for papers of any sort works. For reference: I did my first year of law school at BUSL so i'm very familiar with how it all works there :) This is also very common elsewhere - everything that IBM used to release got tagged with a technical report number too, for example, whether it was or not. In any case - it is clearly a piece meant to be persuasive writing, rather than deep research. Law journals contain a mix of essentially op-eds and deeper research papers or factual expositories/kind of thing. They are mostly not like scientific journals. Though some exist that are basically all op-ed or zero op-ed. Compare something like: https://repository.uclawsf.edu/cgi/viewcontent.cgi?article=2351&context=hastings_law_journal https://repository.uclawsf.edu/cgi/viewcontent.cgi?article=2... Which is a piece in UC law journal meant as an informative piece cataloguing how california courts adjudicate false advertising law. It does not really take a position. with https://repository.uclawsf.edu/cgi/viewcontent.cgi?article=3020&context=hastings_law_journal https://repository.uclawsf.edu/cgi/viewcontent.cgi?article=3... Which is a piece in UC hastings law journal meant as, essentially an op ed, arguing that dog sniff tests are bullshit. I picked both of these at random from stuff in UC hastings law journal that had been cited by the Supreme Court of California. There are things that are even more factual/take zero positions, and things that are even more persuasive writing/less researchy, than either of these, but they are reasonable representatives, i think

charcircuit 9mo ago

None of these paper's arguments are AI specific. The IRS doesn't need AI to make mistakes and be unable to tell you why it did so. You can find stories of that happening to people already.

toofy 9mo ago
i think when most people bring up mistakes that these models make, much of their concern is that little can be done. when one of the juniors makes a mistake, i can talk to them about it and help them understand where they went wrong, if they continue to make mistakes we can change their position to something more suited for them. we can always let them go if they have too much hubris to learn. who do we hold to account when a model makes a mistake? we’re already beginning to see, after major fuckups, companies blackhole nullrouting accountability into “not our fault, don’t look at us, ai was wrong” the other thing is, if you have done a good job selecting your team, you’ll have people who understand their limits, who understand when to ask for help, who understand when they don’t know something. a major problem with current models is that it will always just guess or stretch toward random rather than halt. so yes, people will make mistakes, but at least you can count on being able to mitigate for those after.
- chrisjj 9mo ago
  > who do we hold to account when a model makes a mistake? First we stop anthromorphising the program as capable of making a "mistake". We recognise it merely as machine providing incorrect output, so we see the only mistake was made by the human who chose to rely upon it. The courts so far agree. Judges are punishing the gulled lawyers rather than their faux-intelligent tools.
- terminalshort 9mo ago
  Who was held to account when the IRS made a mistake and sent me a demand letter for over $100K of "unpaid taxes" I didn't owe? Who compensated me for the hours I spent on hold and the money I had to pay an accountant to deal with it?

qsera 9mo ago

We should be more worried what AI will due to the ability of an average human to think. Not that I think there is a lot of thinking going on now anyway, thanks to our beloved smartphones. But just think about a time when human ability to reason has atrophied globally. AI might even give us true Idiocracy!

fennecbutt 9mo ago
I mean we were seeing this even before AI. It's the same type of person. To slop is human. It's like for some reason we thought that like some good percentage of us aren't just tribal worker drones who fundamentally just want fats, sugars, salts, dopamine and seratonin. People actively vote against things like UBI, higher corporate taxes, making utilities public. People actively choose to believe misinformation because it suits their own personal tribal narratives.
t0bia_s 9mo ago
Similar rhetoric was allways there with new technologies. Calculators, radio, cameras, phones, computers, smartphones, social networks... Regulations do more harm than learning process from mistakes.
- getnewmaterial 9mo ago
  Yes we have this before and everytime it was correct.
  - t0bia_s 9mo ago
    What was correct? That we became idiots with new tools?
    - qsera 9mo ago
      >That we became idiots with new tools? Yes, a little bit each time. But AI will finish the job. Because earlier it was writing skills, or attention span that was at stake. This time it is literally the ability to think.

juggle-anyhow 9mo ago

Who do institutions serve? To me AI democratises information. Allows access to information that would normally be gatekept. AI reduces barriers, and they don't like that because those barriers gave them authority.

magpi3 9mo ago

> Purpose-driven institutions built around transparency, cooperation, and accountability empower individuals to take intellectual risks and challenge the status quo. I am not sure if I am off-topic, but I am having a lot of trouble with this statement. Institutions are often opaque, and I have never belonged to an institution that empowered me to "take intellectual risks and challenge the status quo." Quite the contrary.

popalchemist 9mo ago
"purpose-driven" is the relevant qualifier here.

intended 9mo ago

I fear the title of this article is going drive most of the conversation. I haven’t read through the whole thing yet, but so far the parts of the argument I can pull out are about how Institutions actually work, as in a collection of humans. AI, as it currently stands, interacts with humans themselves in ways that hollow out the kind of behavior we want from institutions. “ Perhaps if human nature were a little less vulnerable to the siren’s call of shortcuts, then AI could achieve the potential its creators envisioned for it. But that is not the world we live in. Short-term political and financial incentives amplify the worst aspects of AI systems, including domination of human will, abrogation of accountability, delegation of responsibility, and obfuscation of knowledge and control” An analogy that I find increasingly useful is that of someone using a forklift to lift weights at gym. There is an observable tendency when using LLMs, to cede agency entirely to the machine.

chrisjj 9mo ago
> Perhaps if human nature were a little less vulnerable to the siren’s call of shortcuts, then AI could achieve the potential its creators envisioned for it. I can't see how, given that potential is 99% shorcuts.

layer8 9mo ago

I was amused at how they quote War Games.

incomingpain 9mo ago

>Civic institutions—the rule of law, universities, and a free press—are the backbone of democratic life. They are the mechanisms through which complex societies encourage cooperation and stability, while also adapting to changing circumstances. https://www.pewresearch.org/politics/2025/12/04/public-trust-in-government-1958-2025/ https://www.pewresearch.org/politics/2025/12/04/public-trust... In the 1960s, trust in institutions was around 70%. In 2025 it's about 17%. This is not something like we had 70% until 2023 and then AI dropped our trust suddenly. If anything, AI doesnt even register on the graph. So correlation here is practically non-existent. Gallup and Pew have the similar trends for journalists and universities. You dont get to blame AI for this. Interesting bump and question. How did clinton improve reputation and then bush destroy it? Or is that a false hump?

terminalshort 9mo ago
It's not a false hump, but why do you assume the president caused it just because it happened at the same time?

cornholio 9mo ago

I would summarize the central claim of the paper as: the widespread use of AI to mediate human interaction will rob people of agency, understanding and skill development, as well as destroying the social links necessary to maintain and improve institutions, while at the same time allowing powerful unaccountable actors (AI cabal) to interject into those relations and impose their institutional goals; by "institution" we mean a shared set of beneficial social rules, not merely an organization tasked with promoting them, "justice" vs. "US justice system". The authors then break down the mechanisms by which AI achieves these outcomes (that seem quite reductive and dated compared to the frontier, for example they take it as granted that AI cannot be creative, that it can only work prospectively and can't react to new situations and events etc.), as well as exemplifying those mechanism already at work in a few areas like journalism and academia.

palmotea 9mo ago
And I think that's about right. Despite the marketing, I think AI (especially if the hyped capabilities arrive) will be one of the most destructive technologies ever invented. It only looks good to blinkered and deluded technocrats.
_DeadFred_ 9mo ago
AI is by it's nature an entropy machine.

reaperducer 9mo ago

AI Destroys Institutions Working as intended. WONTFIX.

roxolotl 9mo ago

This articles claims are also interesting to me in terms of large scale software systems. They authors say: > They (AI systems) delegitimization knowledge, inhibit cognitive development, short-circuit long term thinking processes, and isolate humans by displacing or degrading long term human connection. This is a pretty good summary of the worry I’ve seen expressed about extensive use to build large pieces of software. Large pieces of software aren’t just the code that describes them. They exist in some sense in their authors as well. Effective projects seek to expand understanding of software systems across the organization so that relevant decisions can be made about their future. By relegating much of the decision making around structure of the software to AI you lose the systemic knowledge shared across the organization.

TYPE_FASTER 9mo ago

> The affordances of AI systems have the effect of eroding expertise Recently, it so happened that I spent an hour reverse engineering and documenting a piece of a system. A co-worker asked a LLM to do the same. It generated some really nice documentation. The difference is, I (as a team member) now have the understanding. Generating the documentation does not increase the understanding of the team.

ChrisArchitect 9mo ago

Related: How Generative AI is destroying society https://garymarcus.substack.com/p/how-generative-ai-is-destroying-society https://garymarcus.substack.com/p/how-generative-ai-is-destr... (https://news.ycombinator.com/item?id=46616713 https://news.ycombinator.com/item?id=46616713)

mapotofu 9mo ago

I think the argument makes a central mistake in putting too much trust into institutions. I don’t disagree with the conclusions, but the premise around blindly trusting institutions simply because they’ve been around for a long time lost me from taking most of their arguments seriously, despite opening the article thinking I would agree.

notanastronaut 9mo ago

"Abundance of books makes men less studious" - 15th-century Venetian editor, Hieronimo Squarciafico

tim333 9mo ago

The arguments that "AI Destroys Institutions" seem pretty iffy. I scrolled down to see what institutions had been destroyed it went on about Elon Musk's Doge destroying stuff but their closing say USAID I think had zero to do with AI. I'm in the UK and I don't think any institutions have been destroyed or even noticeably harmed by AI. In the US there is general chaos under Trump so it may be hard to differentiate.

i-blis 9mo ago

> Civic institutions—the rule of law, universities, and a free press—are the backbone of democratic life. I disagree. The backbone of democratic life are the rule of law and freedom of speech, which makes a big difference. The press has historically been a counter-power inquiring into privileges and breaches of the rule of the law and thus promoted freedom of speech but almost only inasmuch it served the interest of the emerging merchant bourgeoisie . And we are long past that. Universities never have been liberal forces: they backed the Church and refused paradigm shifts. They still are very conservative even though in a peculiar sense, as leftist conservatives.

MohskiBroskiAI 9mo ago

[flagged]