7 ms·

AI-assisted cognition endangers human development?

230 points by i5heu 6mo ago

blackqueeriroh 6mo ago
This is bad science. Horrifically bad science.
bomewish 6mo ago
Doh. I went in expecting a really cool thesis — because the idea seems somehow intuitive, or at least really intriguing. But I have no clue what I read. Just totally odd and unconvincing. Greenland? Dialectal substrate? The idea is still super intriguing to me though!
- chromacity 6mo ago
  Well, at least you know it's not AI-written because it's delightfully weird and evidently about some pet theory of the author. This day and age, that's something to unironically celebrate.
  - ulf-77723 6mo ago
    I love this! Especially the part about greenland. For quite some time the dashes were a good indicator of a text was written by AI - but now the best option is to write more human like by doing it a little less polished but weird - at least the message is being transported
  - ACCount37 6mo ago
    A lot of (allegedly human) cranks now bounce their wacky ideas off a "you're completely right" LLM. AI psychosis has been going through armchair philosophers, physicists and political theorists the way crack was going through the low income neighborhoods back in the 80s.
- asdfman123 6mo ago
  While I understand what the paper is saying I'm not sure if what I read was written by someone who is smarter than me and naturally goes higher up the abstraction tree, or just wants to write really smart things. Either way though I think there's a much simpler way to express what she's trying to say. Offloading thinking to AI is bad because it's less flexible and doesn't easily update its reasoning with new information.
  - layer8 6mo ago
    It’s a blog post, not a paper.
- GeoAtreides 6mo ago
  >I have no clue what I read. Just totally odd and unconvincing. How can it be unconvincing if you didn't understand the argument presented in the post?
steve_adams_86 6mo ago
"Cognitive inbreeding" is an interesting (though maybe not entirely accurate) term for something I dislike a lot about LLMs. It really is a thing. You're recycling the same biases over and over, and it can be very difficult to tell if you don't review and distill the contents of your discourse with LLMs. Especially true if you're only using one. I do think there's a solution to this—kind of—which dramatically reduces the probability and allowing for broad inductive biases. And that's to ask question with narrower scopes, and to ensure you're the one driving conversation. It's true with programming as well. When you clearly define what you need and how things should be done, the biases are less evident. When you ask broad questions and only define desired outcomes in ambiguous terms, biases will be more likely to take over. When people ask LLMs to build the world, they will do it in extremely biased ways. This makes sense. When you ask it specifics about narrow topics, this is still be a problem, but greatly mitigated. I suppose what's happening is an inversion of cognitive load, so the human is taking on more and selecting bias such that the LLM is less free to do so. This is roughly in line with the article's premise (maybe not the entire article, though), which is fine; I think I generally agree that these are cognitive muscles that need exercising, and allowing an LLM to do it all for you is potentially harmful. But I don't think we're trapped with the outcome, we do have agency, and with care it's a technology that can be quite beneficial.
- Retr0id 6mo ago
  One of my "let's try out this vibecoding thing" toy projects was a custom programming language. At the time, I felt like it was my design, which I iterated on through collaborative conversations with Claude. Then I saw someone's Show HN post for their own vibecoded programming language project, and many of the feature bullet points were the same. Maybe it was partly coincidence (all modern PLs have a fair bit of overlap), but it really gave me pause, and I mostly lost interest in the project after that.
  - Ucalegon 6mo ago
    Thats the thing about a normalization system, it is going to normalize outputs because its not built to output uniqueness, its to winnow uniqueness to a baseline. That is good in some instances, assuming that baseline is correct, but it also closes the aperture of human expression.
cowlby 6mo ago
Sometimes it feels like as developers we live in a a bubble. Don't most jobs endanger human development? I can't help but think about all the billions of factory, food service, assembly line type jobs. Do these not threaten "human development"? My cynical take would be all AI endangers is "white collar" work.
- godot 6mo ago
  I think you're not wrong and I also think the author is not wrong -- and this just may be how technology/civilization/humans are going to change inevitably? For example a possibly trajectory might be that many years in the future because human thinking has degraded due to AI-assisted cognition, most people will get a chip implant and AI-assistance becomes integrated with the brain. Basically same pattern as most everything else -- technology augments solve the new reality. I'm not saying this will happen, but just a possible outcome of this.
  - geitir 6mo ago
    Doesn’t that sound lovely
SegfaultSeagull 6mo ago
It’s a bit ironic that the author includes an AI generated audio version of the article, you know, so we don’t have to read it.
- yakattak 6mo ago
  Sounds great for people who are seeing impaired.
  - BigTTYGothGF 6mo ago
    Don't those people tend to have their own setups to do that sort of thing?
    - yakattak 6mo ago
      They do, though I imagine just having “someone” reading it to you with intonation and whatnot is nice. I don’t know though.
  - Argonaut998 6mo ago
    That's what screenreaders are for
  - layer8 6mo ago
    Or driving. Or working around the house.
chunky1994 6mo ago
Does anyone use LLMs in such a manner that they believe it always has the most up to date information (without web search tools?). Isn't this whole thesis negated by the fact that tool calling web search exists? This just feels like a whole lot of words to say, don't treat a LLM as an always up to date infallible statistical predictor.
- karmakurtisaani 6mo ago
  > Does anyone use LLMs in such a manner that they believe it always has the most up to date information (without web search tools?). Probably just 95% of the users. You know, the non-techies.
  - bookofjoe 6mo ago
    https://archive.ph/YwpVw https://archive.ph/YwpVw
  - Planktonne 6mo ago
    The AI hype and overstatement of capabilities is at least as strong amongst the 'techies' as the people they treat as more credulous than themselves.
- amluto 6mo ago
  Why do you expect web search tool calls to continue to be useful in the presence of modern AI slop farms, AI-assisted SEO, and search engines largely turning themselves into AI-based question-answering engines? (At present, Gemini's question-answering capability (which Google kind of makes its users use) seems extremely error-prone -- much worse than competing LLMs when asked the same question.)
  - fl4regun 6mo ago
    I agree with you, this is a huge concern, and we are still in an age where most content on the internet isn't ai generated yet. What about 10 years from now? We have many instances of people writing posts on reddit or uploading videos and blogs using AI generated text. What happens when that is a significant percentage of content? I recently saw a video discussing a researcher who published a fake scientific article about a fictitious disease, with bogus author names, even a warning IN the article itself that stated "This is not a real disease, this article is not real" (paraphrasing) but still AI ended up picking up this article and serving information from it as if it was a real disease. It even got cited in papers (which were later redacted of course), but the fact those papers got published in the first place is a serious issue.
zozbot234 6mo ago
At the Egyptian city of Naucratis, there was a famous old god, whose name was Theuth; the bird which is called the Ibis is sacred to him, and he was the inventor of many arts, such as arithmetic and calculation and geometry and astronomy and draughts and dice, but his great discovery was the use of letters. Now in those days the god Thamus was the king of the whole country of Egypt; and he dwelt in that great city of Upper Egypt which the Hellenes call Egyptian Thebes, and the god himself is called by them Ammon. To him came Theuth and showed his inventions, desiring that the other Egyptians might be allowed to have the benefit of them; he enumerated them, and Thamus enquired about their several uses, and praised some of them and censured others, as he approved or disapproved of them. It would take a long time to repeat all that Thamus said to Theuth in praise or blame of the various arts. But when they came to letters, This, said Theuth, will make the Egyptians wiser and give them better memories; it is a specific both for the memory and for the wit. Thamus replied: O most ingenious Theuth, the parent or inventor of an art is not always the best judge of the utility or inutility of his own inventions to the users of them. And in this instance, you who are the father of letters, from a paternal love of your own children have been led to attribute to them a quality which they cannot have; for this discovery of yours will create forgetfulness in the learners' souls, because they will not use their memories; they will trust to the external written characters and not remember of themselves. The specific which you have discovered is an aid not to memory, but to reminiscence, and you give your disciples not truth, but only the semblance of truth; they will be hearers of many things and will have learned nothing; they will appear to be omniscient and will generally know nothing; they will be tiresome company, having the show of wisdom without the reality.
- alwa 6mo ago
  (From Socrates’ dialog with Phaedrus) https://classics.mit.edu/Plato/phaedrus.html#:~:text=there%20was%20a%20famous%20old%20god%2C https://classics.mit.edu/Plato/phaedrus.html#:~:text=there%2...
  - xg15 6mo ago
    > Phaedrus: Yes, Socrates, you can easily invent tales of Egypt, or of any other country. Looks like even back then, they went "cool story bro" on that text...
LetsGetTechnicl 6mo ago
Well no shit
jbethune 6mo ago
This was a bit word-salad-y but I share the same basic concern. I think more I worry about the tendency toward greater and greater cognitive off-loading to LLMs. My sister told me a story the other day about how she caught her plumber using chatgpt on his phone to fix an issue with her bathroom. I just think it's good for humans to know how to do stuff.
- dfee 6mo ago
  your sister offloaded to her plumber. her plumber offloaded to chatgpt. "i just think it's good for humans to know how to do stuff." are we talking about your sister or her plumber?
  - jessetemp 6mo ago
    The plumber obviously. Not everyone needs to know how to be a plumber, but a plumber should know how to be a plumber
    - dfee 6mo ago
      the question was rhetorical. but, since you responded – do you think that there are limits to who can or should use ai? if the plumber's use of ChatGPT improved outcomes, isn't that preferable? some knowledge is likely "cached" in the plumber. maybe he doesn't ask the same question twice. i'm sympathetic to the plumber, but i think your concerns of erosion of knowledge or skill are worth pushing on further.
      - jessetemp 6mo ago
        > do you think that there are limits to who can or should use ai? I don't think there should be imposed limits, but there might be an upper bound where expertise becomes atrophied by depending on AI too much. > if the plumber's use of ChatGPT improved outcomes, isn't that preferable? In the short term sure, and maybe even in the long term for the customer. I think the risk to the plumber is losing some of their expertise by outsourcing to AI. But who knows, maybe the plumber has excellent memory and only accumulates knowledge each time they use AI. Some of the article is lost in the plumber example. I doubt plumbers are spending much time exploring new ways of solving problems, and might even benefit from having a narrower range of outcomes. Other fields that require both expertise and novel solutions will be at a disadvantage if they become more homogenized by depending on AI. Not only is the range of solutions reduced, but getting there is faster, so people end up in a local maxima. Maybe they get stuck there, maybe not, but that's the risk I see. You don't imagine any long term risks by outsourcing expertise to AI?
throwaway613746 6mo ago
[dead]
thepasch 6mo ago
AI-assisted, I can see. I believe it doesn’t have to be that way, though. If you use AI as a grounding tool - essentially something that can take your stream of consciousness and parse it into a series of concerete and pointed search terms to do real-time research with instead of falling back on what’s in the weights - then it’s honestly hard to think of a technology that had the potential to be more useful in the history of the species - it gives you much more direct access to both your unknown unknowns and your unknown knowns. That is, of course, provided that you pay attention it actually does research. In their current state, LLMs are practically useless for this purpose for the vast majority of users, as no one knows how they work, what to watch out for, what the failure modes look like, and how to keep nonsense apart from facts when both are presented with an equal amount of conviction. That’s not a user problem, it’s an education problem.
demorro 6mo ago
This Dynamic Dialectical Substrate sounds a lot like Pirsigs Metaphysics of Quality to me, which I think is neat.
MillionOClock 6mo ago
Say someone uses AI, treating it as if it was a developer (probably not recommended today due to the risk of errors), and working and speaking with it as if they were some kind of product manager or senior engineer who only makes architectural decisions etc. I wonder what kind of difference would it really make? Sure the person might not be as good anymore as a developer, but how is this different from being a usual product manager or whatever the day AI truly is good enough for a developer role? I'm not saying I know what the answer to this question is, but this is something I genuinely wonder, and I think the same kind of questioning can apply to broader domains.
- kmaitreys 6mo ago
  Why and how do you think it applies to broader domains? Children learning in schools should not become product managers. If they are, what exactly is the "product" that they are "managing"? Reducing everything to and looking everything from a corporate viewpoint is bizarre.
  - MillionOClock 6mo ago
    I'm not saying this should be every single domain. This isn't about products or management, instead I would frame it like this: I notice that multiple cases where we are worried about the impact of AI are basically just about the replacement of certain activities that some humans already aren't doing in today's society. If we are worried we will be less good at doing job X once we don't do job X anymore, why are we not worried about people who never did job X in the first place? If we are worried about people not doing jobs anymore, why are not worried for the human development of people wealthy enough not to work anymore for the rest of their days? I would not assume someone who won the lottery is going to have their life become uninteresting or see some cognitive decline. It could probably happen, but you can also see a path where the person just chooses to do the activities they always wanted to do, where they keep learning and exploring without the burden of usual life constraints. People already play chess when machines have beaten us for decades, just because they enjoy it. Regarding education I think AI is a huge revolution waiting to happen. Usual courses have become boring? Have future super powerful AI generate per student highly personalized programs, create bespoke video games where succeeding can only happen once the student has validated all the notions you wanted them to validate etc.
YackerLose 6mo ago
A real artificial intelligence would be capable of independent and original thought. What we have today are mere plagiarism factories. They need to be called out for what they are.
greatpost 6mo ago
[dead]
waffletower 6mo ago
[dead]
kazinator 6mo ago
> Speaking and discussing with other humans [who aren't incessantly blathering about AI] is obviously the most effective way to mitigate these problems. Slightly FTFY.
adamtaylor_13 6mo ago
One thing that's always been true with human communication that is becoming increasingly obvious to me through my interactions with LLMs is the art of asking a good question. The framing of questions massively affects the results you get from discussion with humans, and I'd argue it's even more pronounced with LLMs.
- alfalfasprout 6mo ago
  Yep. And this is why as hard as AI companies are pushing that these tools can be a replacement for expertise, it's ironic that the experts are the ones that often get the highest ROI because they know how to converse about the relevant subjects with a high degree of precision (and know what to look for, what to challenge, etc.).
  - samxli 6mo ago
    This is also why we need to teach critical thinking skills better in schools so future humans can use AI more effectively.
gobdovan 6mo ago
By the logic that today's news is fundamental to know as true, there really is no point in reading books older than 6 months old. If Einstein woke up from a coma, he'd be useless, as he doesn't even know who won the World Cup. For real now, if an AI can help you solve a problem using 2,000 years of human logic, does it really matter if it's "skewed" away from a political shift that happened three weeks ago? I also don't believe that everybody I know is idiosyncratic in the way they view the world. And even if they were, I'd probably just pay attention to the things that are directly relevant to me. So probably I'll misunderstand most of what they say anyway.
- Yokohiii 6mo ago
  The post doesn't question the value of historic knowledge. The idea is that a base model could be years old, with all it's faults; superficially fixed and extended with knowledge over time. She points to a research that LLMs don't "fully believe" this new knowledge. The skew could be much longer then a few weeks.
measurablefunc 6mo ago
Calculators endanger the development of mental arithmetic skills as well.
- layer8 6mo ago
  Indeed, which is why it’s preferable to only start using them after some arithmetic maturity is achieved.
- add-sub-mul-div 6mo ago
  Right, which is why people make bad money decisions in everyday scenarios. People don't pull out their calculator at the grocery store, but they also never had to get good at doing simple math in their head due to the calculator.
- jojomodding 6mo ago
  And we collectively decided that it's fine, you don't actually need to be able to solve 1234×5678 in your head. But I am not sure you can compartmentalize the specific skill we can out-source to AI. I would not agree with "you don't need to be able to think in your head."
toooomato 6mo ago
[dead]
svnt 6mo ago
It is a quirky article but the author, instead of engaging with information sources to understand what important thoughts people have had about these topics, feels the best thing to do is introduce new terms that other terms already exist for. This is basically just inductive bias plus the AI homogenization idea producing a distribution shift. This is what happens in thought-isolation. It isn’t better than educating yourself, whether that education involves AI or not. Phillip Kitcher is known for epistemic monoculture, Dawkins and then Henrich popularized collective intelligence and cultural evolution. The thing about these fear pieces is concepts like the hollowed mind are reductive and that reductionism is based on a reductive view of (usually other) people. But what actually happens is we have formalized processes and can externalize them. This is a benefit if you can use your newfound capacity and free time for something better, which I think most people ultimately will.
- Forgeties79 6mo ago
  >This is a benefit if you can use your newfound capacity and free time for something better, which I think most people ultimately will. I think for a lot of of us the problem is that this is not a given. It’s often promised and rarely occurs, especially in the modern era. Increased productivity usually just means increased demands in the workplace.
  - svnt 6mo ago
    Yeah I can’t disagree I think it is really disruptive in a corporate-job-expectation setting where you don’t have the same level of agency.
  - le-mark 6mo ago
    Similar to What people were saying about television. They imagined it would bring Shakespeare to the masses, but what ultimately ended up happening is television met people where they are hence we got reality TV. There’s no reason to believe AI won’t be similar.
    - vlunkr 6mo ago
contingencies 6mo ago
Strong disagree. The "AI-Assisted Cognition" phrase is loaded. Would you attempt to, for example, simultaneously modify for available ingredients, number of diners, and time-optimize the prep method for a recipe you've never cooked before if you were following an old-school cookbook? No. You'd have to be a pretty solid chef to try all that on at once. Using AI, you might branch out confidently in to new areas, executing all of these modifications simultaneously, and even adapting the output for a specific audience or language. This toy example shows an important property of AI as decision support systems, which are well studied in the military domain: using these systems, we build confidence to act in unfamiliar domains, thereby extending our reach. From this experience we can learn more. The fact that the learning may then occur through, ie. during or after the experience, rather than beforehand, is secondary. It's still there. The fact we didn't know the language the AI translated to for our chef is totally irrelevant. Sitting comfortably at the effective apex of millions of years of human cognitive and technology development with the entire world's knowledge at our fingertips, every day we can extend confidence in novel domains through AI, and enjoy it. We should be feeling pretty damn "developed". Rote formalism and fixed paths in pedagogy are gone: good riddance. This is the hacker age.
Manuel_D 6mo ago
> In early 2026, the USA prepared to invade Greenland and, therefore, the EU4. Only a few months prior to that it was completely unthinkable that the USA would even think about threatening an invasion of Greenland. As AI base models are stuck in the past, they do not easily accept these events as real and often label them as “hypothetical”, “fake news”, or “impossible”. This also affects new models like Gemini 3 Pro, GLM-5 or GPT-5.3-codex5. Isn't this just inherent to any system that takes some time to update? E.g. if a country moves its capital to a different city, then textbooks, maps, etc. are going to contain incorrect information for a while until updated editions are published. A lot of the complaints about AI are really about the drawbacks of information systems more generally, and the failure modes pointed out are rarely novel. The "Cognitive Inbreeding" effect attributed to AI would also have occurred with Google search would it not? Lots of people type the same question into google and read the top results, instead of searching a more diverse set of information sources. It's interesting that the author mentions web search as a way to ameliorate this, when it seems to me that web search is just as capable of causing cognitive inbreeding.
- djrorkrmrk 6mo ago
  A few years ago, someone blow up a pipeline in eu. Before thwt some people lied about medical stuff. AI is just current scapegoat.
- darepublic 6mo ago
  I agree that Google kind of serves this role even before llms. But these days people delegate their reasoning, brainstorming to the computer not just lookup. And beyond our generation are those who would have grown up doing this. Therefore I think concern is justified
- Yokohiii 6mo ago
  In school we were often required to have a specific edition, or we got notice about a certain thing that changed. People that relied on print knew that it could be wrong. Deliberate changes like a name change, will lead to errors, everybody expects and deals with that. In digital space we expect that to be rather unlikely, at least for major maintained sources. I think the difference is that LLMs are a very complex mix of information and concepts, which can be combined in higher orders. So an underlying wrong fact could be undisclosed and contribute to faulty reasoning. A hard fact like a wrong city name would blow up quickly. A wrong assumption about political dynamics is probably harder to detect, as it is a complex mix of information.
dcre 6mo ago
I've never seen an argument like this that, if true, wouldn't also apply to the cognitive offloading we do by relying on culture, by working with others, or working with the artifacts built by others.
- 0xBA5ED 6mo ago
  Cognitive offloading via culture has many forms and many of them are not sustainable at all.
  - dcre 6mo ago
    Sure! I don't mean they're all good. I just mean that it can't be cognitive offloading itself that is the problem, but the particular character of it.
darepublic 6mo ago
The original "person who most of humanity talked to" was, I reckon, google dot com
drusepth 6mo ago
This is absolutely something to potentially be worried about, but one thing I never see highlighted in critiques of AI-assisted cognition is that some elements of physiology may not actually be biologically necessary if they can be fully supplanted by some replacement (in this case, new tools). I can't traverse as much land on foot as my ancestors did (my muscles are weaker, my endurance is less, etc), but I can travel even further than they could by car/plane/etc. Nothing about the nature of evolution implies our current cognitive processing is ideal/sacred and shouldn't ever change.
- lexandstuff 6mo ago
  > I can't traverse as much land on foot as my ancestors did, but I can travel further by car/plane/etc Which is partially how we found ourselves in the midst of an obesity epidemic.
  - ToValueFunfetti 6mo ago
    Only either indirectly or extremely partially. Everybody was driving by the 40s in the US but the obesity epidemic didn't start until the late 70s. Really, obesity is an example of exactly what the GP was talking about- our evolved relationship with food is not inherently good, and it's better for us to change our behaviors than to abandon our advancements and return to the food-scarce world we're adapted to.
    - jplusequalt 6mo ago
      >our evolved relationship with food is not inherently good, and it's better for us to change our behaviors than to abandon our advancements and return to the food-scarce world we're adapted to. So are you arguing we should change our relationship with human intelligence? What does that even mean?
      - ToValueFunfetti 6mo ago
        My intent was to argue that the obesity epidemic is not a supporting analogy for their point. I didn't mean to imply that the problems with it analogize back to AI.
mayankd 6mo ago
The cognitive effects are going to be so divergent. While the avid learners will learn knowledge and skills on the fly exponentially faster, the populace offloading thinking to the AI models will see unprecedented cognitive decline. This is similar to the effect that the internet had on knowledge retention but this time on critical thinking
cyanydeez 6mo ago
Do we think AI is similar to being rich, but without all that cash? I mean, they can basically offload most things to other people to think about.
drivebyhooting 6mo ago
If computers are bicycles for the mind and AI are cars, I wonder what the analogue for the obesity epidemic is.
- gdubya 6mo ago
  That is a sharp and slightly chilling analogy. If Steve Jobs saw the computer as a tool that amplified human effort (the bicycle), and AI represents a tool that automates that effort entirely (the car), then the "obesity epidemic" of the mind is likely Cognitive Atrophy. - Gemini
  - antonvs 6mo ago
    > That is a sharp LLM tell right there. > - Gemini Yes, we already know. I suppose you think posting AI slop in this context is funny. It isn't. Also, no, the observation is not sharp. You're being gaslighted and having your cock fluffed by a machine.
- antonvs 6mo ago
  The obesity epidemic has much less to do with cars and much more to do with cheapness of food and volume consumed. A typical deli sandwich in the US should be enough to last any normal person three days. Same goes for e.g. ice cream from Shake Shack (random example I know, but one I came across recently). If you buy one of these and eat them in one sitting, the answer to "why am I obese" is simply "you eat way too much."
  - cyberax 6mo ago
    Sorry, don't exaggerate please. A Subway sandwhich is about 600 calories. That's about 1/3 of the daily 2000 calories standard. A Shake Shack shake tops out at 1010 calories, a half of the daily norm.
    - antonvs 6mo ago
      I'm not talking about Subway. I specifically said deli sandwich. There's no world in which Subway is a deli. As for Shake Shack, a single shake is half of the daily total calorie intake? Are you listening to yourself right now?
giancarlostoro 6mo ago
I think the best way I can put it is probably; this is the same as if you just cheat off someone else in school, you aren't learning much are you? AI is the same thing. Don't just cheat, use it to learn instead.
anigbrowl 6mo ago
So does talking to uninformed people. The size of the group is inversely correlated with deviation from the mean (of IQ, productivity, or whatever proxy for cognitive capability you care to specify). I'm not sure why this is at the top of the page; it's not that it's wrong, it's just a sequence of truisms.
2OEH8eoCRo0 6mo ago
Economic incentives are forcing us to use tools that make us dumber. What does the future hold?

230 points by i5heu 6mo ago

This is bad science. Horrifically bad science.

Doh. I went in expecting a really cool thesis — because the idea seems somehow intuitive, or at least really intriguing. But I have no clue what I read. Just totally odd and unconvincing. Greenland? Dialectal substrate? The idea is still super intriguing to me though!

chromacity 6mo ago
Well, at least you know it's not AI-written because it's delightfully weird and evidently about some pet theory of the author. This day and age, that's something to unironically celebrate.
- ulf-77723 6mo ago
  I love this! Especially the part about greenland. For quite some time the dashes were a good indicator of a text was written by AI - but now the best option is to write more human like by doing it a little less polished but weird - at least the message is being transported
- ACCount37 6mo ago
  A lot of (allegedly human) cranks now bounce their wacky ideas off a "you're completely right" LLM. AI psychosis has been going through armchair philosophers, physicists and political theorists the way crack was going through the low income neighborhoods back in the 80s.
asdfman123 6mo ago
While I understand what the paper is saying I'm not sure if what I read was written by someone who is smarter than me and naturally goes higher up the abstraction tree, or just wants to write really smart things. Either way though I think there's a much simpler way to express what she's trying to say. Offloading thinking to AI is bad because it's less flexible and doesn't easily update its reasoning with new information.
- layer8 6mo ago
  It’s a blog post, not a paper.
GeoAtreides 6mo ago
>I have no clue what I read. Just totally odd and unconvincing. How can it be unconvincing if you didn't understand the argument presented in the post?

steve_adams_86 6mo ago

"Cognitive inbreeding" is an interesting (though maybe not entirely accurate) term for something I dislike a lot about LLMs. It really is a thing. You're recycling the same biases over and over, and it can be very difficult to tell if you don't review and distill the contents of your discourse with LLMs. Especially true if you're only using one. I do think there's a solution to this—kind of—which dramatically reduces the probability and allowing for broad inductive biases. And that's to ask question with narrower scopes, and to ensure you're the one driving conversation. It's true with programming as well. When you clearly define what you need and how things should be done, the biases are less evident. When you ask broad questions and only define desired outcomes in ambiguous terms, biases will be more likely to take over. When people ask LLMs to build the world, they will do it in extremely biased ways. This makes sense. When you ask it specifics about narrow topics, this is still be a problem, but greatly mitigated. I suppose what's happening is an inversion of cognitive load, so the human is taking on more and selecting bias such that the LLM is less free to do so. This is roughly in line with the article's premise (maybe not the entire article, though), which is fine; I think I generally agree that these are cognitive muscles that need exercising, and allowing an LLM to do it all for you is potentially harmful. But I don't think we're trapped with the outcome, we do have agency, and with care it's a technology that can be quite beneficial.

Retr0id 6mo ago
One of my "let's try out this vibecoding thing" toy projects was a custom programming language. At the time, I felt like it was my design, which I iterated on through collaborative conversations with Claude. Then I saw someone's Show HN post for their own vibecoded programming language project, and many of the feature bullet points were the same. Maybe it was partly coincidence (all modern PLs have a fair bit of overlap), but it really gave me pause, and I mostly lost interest in the project after that.
- Ucalegon 6mo ago
  Thats the thing about a normalization system, it is going to normalize outputs because its not built to output uniqueness, its to winnow uniqueness to a baseline. That is good in some instances, assuming that baseline is correct, but it also closes the aperture of human expression.

cowlby 6mo ago

Sometimes it feels like as developers we live in a a bubble. Don't most jobs endanger human development? I can't help but think about all the billions of factory, food service, assembly line type jobs. Do these not threaten "human development"? My cynical take would be all AI endangers is "white collar" work.

godot 6mo ago
I think you're not wrong and I also think the author is not wrong -- and this just may be how technology/civilization/humans are going to change inevitably? For example a possibly trajectory might be that many years in the future because human thinking has degraded due to AI-assisted cognition, most people will get a chip implant and AI-assistance becomes integrated with the brain. Basically same pattern as most everything else -- technology augments solve the new reality. I'm not saying this will happen, but just a possible outcome of this.
- geitir 6mo ago
  Doesn’t that sound lovely

SegfaultSeagull 6mo ago

It’s a bit ironic that the author includes an AI generated audio version of the article, you know, so we don’t have to read it.

yakattak 6mo ago
Sounds great for people who are seeing impaired.
- BigTTYGothGF 6mo ago
  Don't those people tend to have their own setups to do that sort of thing?
  - yakattak 6mo ago
    They do, though I imagine just having “someone” reading it to you with intonation and whatnot is nice. I don’t know though.
- Argonaut998 6mo ago
  That's what screenreaders are for
- layer8 6mo ago
  Or driving. Or working around the house.

chunky1994 6mo ago

Does anyone use LLMs in such a manner that they believe it always has the most up to date information (without web search tools?). Isn't this whole thesis negated by the fact that tool calling web search exists? This just feels like a whole lot of words to say, don't treat a LLM as an always up to date infallible statistical predictor.

karmakurtisaani 6mo ago
> Does anyone use LLMs in such a manner that they believe it always has the most up to date information (without web search tools?). Probably just 95% of the users. You know, the non-techies.
- bookofjoe 6mo ago
  https://archive.ph/YwpVw https://archive.ph/YwpVw
- Planktonne 6mo ago
  The AI hype and overstatement of capabilities is at least as strong amongst the 'techies' as the people they treat as more credulous than themselves.
amluto 6mo ago
Why do you expect web search tool calls to continue to be useful in the presence of modern AI slop farms, AI-assisted SEO, and search engines largely turning themselves into AI-based question-answering engines? (At present, Gemini's question-answering capability (which Google kind of makes its users use) seems extremely error-prone -- much worse than competing LLMs when asked the same question.)
- fl4regun 6mo ago
  I agree with you, this is a huge concern, and we are still in an age where most content on the internet isn't ai generated yet. What about 10 years from now? We have many instances of people writing posts on reddit or uploading videos and blogs using AI generated text. What happens when that is a significant percentage of content? I recently saw a video discussing a researcher who published a fake scientific article about a fictitious disease, with bogus author names, even a warning IN the article itself that stated "This is not a real disease, this article is not real" (paraphrasing) but still AI ended up picking up this article and serving information from it as if it was a real disease. It even got cited in papers (which were later redacted of course), but the fact those papers got published in the first place is a serious issue.

zozbot234 6mo ago

At the Egyptian city of Naucratis, there was a famous old god, whose name was Theuth; the bird which is called the Ibis is sacred to him, and he was the inventor of many arts, such as arithmetic and calculation and geometry and astronomy and draughts and dice, but his great discovery was the use of letters. Now in those days the god Thamus was the king of the whole country of Egypt; and he dwelt in that great city of Upper Egypt which the Hellenes call Egyptian Thebes, and the god himself is called by them Ammon. To him came Theuth and showed his inventions, desiring that the other Egyptians might be allowed to have the benefit of them; he enumerated them, and Thamus enquired about their several uses, and praised some of them and censured others, as he approved or disapproved of them. It would take a long time to repeat all that Thamus said to Theuth in praise or blame of the various arts. But when they came to letters, This, said Theuth, will make the Egyptians wiser and give them better memories; it is a specific both for the memory and for the wit. Thamus replied: O most ingenious Theuth, the parent or inventor of an art is not always the best judge of the utility or inutility of his own inventions to the users of them. And in this instance, you who are the father of letters, from a paternal love of your own children have been led to attribute to them a quality which they cannot have; for this discovery of yours will create forgetfulness in the learners' souls, because they will not use their memories; they will trust to the external written characters and not remember of themselves. The specific which you have discovered is an aid not to memory, but to reminiscence, and you give your disciples not truth, but only the semblance of truth; they will be hearers of many things and will have learned nothing; they will appear to be omniscient and will generally know nothing; they will be tiresome company, having the show of wisdom without the reality.

alwa 6mo ago
(From Socrates’ dialog with Phaedrus) https://classics.mit.edu/Plato/phaedrus.html#:~:text=there%20was%20a%20famous%20old%20god%2C https://classics.mit.edu/Plato/phaedrus.html#:~:text=there%2...
- xg15 6mo ago
  > Phaedrus: Yes, Socrates, you can easily invent tales of Egypt, or of any other country. Looks like even back then, they went "cool story bro" on that text...

LetsGetTechnicl 6mo ago

Well no shit

jbethune 6mo ago

This was a bit word-salad-y but I share the same basic concern. I think more I worry about the tendency toward greater and greater cognitive off-loading to LLMs. My sister told me a story the other day about how she caught her plumber using chatgpt on his phone to fix an issue with her bathroom. I just think it's good for humans to know how to do stuff.

dfee 6mo ago
your sister offloaded to her plumber. her plumber offloaded to chatgpt. "i just think it's good for humans to know how to do stuff." are we talking about your sister or her plumber?
- jessetemp 6mo ago
  The plumber obviously. Not everyone needs to know how to be a plumber, but a plumber should know how to be a plumber
  - dfee 6mo ago
    the question was rhetorical. but, since you responded – do you think that there are limits to who can or should use ai? if the plumber's use of ChatGPT improved outcomes, isn't that preferable? some knowledge is likely "cached" in the plumber. maybe he doesn't ask the same question twice. i'm sympathetic to the plumber, but i think your concerns of erosion of knowledge or skill are worth pushing on further.
    - jessetemp 6mo ago
      > do you think that there are limits to who can or should use ai? I don't think there should be imposed limits, but there might be an upper bound where expertise becomes atrophied by depending on AI too much. > if the plumber's use of ChatGPT improved outcomes, isn't that preferable? In the short term sure, and maybe even in the long term for the customer. I think the risk to the plumber is losing some of their expertise by outsourcing to AI. But who knows, maybe the plumber has excellent memory and only accumulates knowledge each time they use AI. Some of the article is lost in the plumber example. I doubt plumbers are spending much time exploring new ways of solving problems, and might even benefit from having a narrower range of outcomes. Other fields that require both expertise and novel solutions will be at a disadvantage if they become more homogenized by depending on AI. Not only is the range of solutions reduced, but getting there is faster, so people end up in a local maxima. Maybe they get stuck there, maybe not, but that's the risk I see. You don't imagine any long term risks by outsourcing expertise to AI?

throwaway613746 6mo ago

[dead]

thepasch 6mo ago

AI-assisted, I can see. I believe it doesn’t have to be that way, though. If you use AI as a grounding tool - essentially something that can take your stream of consciousness and parse it into a series of concerete and pointed search terms to do real-time research with instead of falling back on what’s in the weights - then it’s honestly hard to think of a technology that had the potential to be more useful in the history of the species - it gives you much more direct access to both your unknown unknowns and your unknown knowns. That is, of course, provided that you pay attention it actually does research. In their current state, LLMs are practically useless for this purpose for the vast majority of users, as no one knows how they work, what to watch out for, what the failure modes look like, and how to keep nonsense apart from facts when both are presented with an equal amount of conviction. That’s not a user problem, it’s an education problem.

demorro 6mo ago

This Dynamic Dialectical Substrate sounds a lot like Pirsigs Metaphysics of Quality to me, which I think is neat.

MillionOClock 6mo ago

Say someone uses AI, treating it as if it was a developer (probably not recommended today due to the risk of errors), and working and speaking with it as if they were some kind of product manager or senior engineer who only makes architectural decisions etc. I wonder what kind of difference would it really make? Sure the person might not be as good anymore as a developer, but how is this different from being a usual product manager or whatever the day AI truly is good enough for a developer role? I'm not saying I know what the answer to this question is, but this is something I genuinely wonder, and I think the same kind of questioning can apply to broader domains.

kmaitreys 6mo ago
Why and how do you think it applies to broader domains? Children learning in schools should not become product managers. If they are, what exactly is the "product" that they are "managing"? Reducing everything to and looking everything from a corporate viewpoint is bizarre.
- MillionOClock 6mo ago
  I'm not saying this should be every single domain. This isn't about products or management, instead I would frame it like this: I notice that multiple cases where we are worried about the impact of AI are basically just about the replacement of certain activities that some humans already aren't doing in today's society. If we are worried we will be less good at doing job X once we don't do job X anymore, why are we not worried about people who never did job X in the first place? If we are worried about people not doing jobs anymore, why are not worried for the human development of people wealthy enough not to work anymore for the rest of their days? I would not assume someone who won the lottery is going to have their life become uninteresting or see some cognitive decline. It could probably happen, but you can also see a path where the person just chooses to do the activities they always wanted to do, where they keep learning and exploring without the burden of usual life constraints. People already play chess when machines have beaten us for decades, just because they enjoy it. Regarding education I think AI is a huge revolution waiting to happen. Usual courses have become boring? Have future super powerful AI generate per student highly personalized programs, create bespoke video games where succeeding can only happen once the student has validated all the notions you wanted them to validate etc.

YackerLose 6mo ago

A real artificial intelligence would be capable of independent and original thought. What we have today are mere plagiarism factories. They need to be called out for what they are.

greatpost 6mo ago

[dead]

waffletower 6mo ago

[dead]

kazinator 6mo ago

> Speaking and discussing with other humans [who aren't incessantly blathering about AI] is obviously the most effective way to mitigate these problems. Slightly FTFY.

adamtaylor_13 6mo ago

One thing that's always been true with human communication that is becoming increasingly obvious to me through my interactions with LLMs is the art of asking a good question. The framing of questions massively affects the results you get from discussion with humans, and I'd argue it's even more pronounced with LLMs.

alfalfasprout 6mo ago
Yep. And this is why as hard as AI companies are pushing that these tools can be a replacement for expertise, it's ironic that the experts are the ones that often get the highest ROI because they know how to converse about the relevant subjects with a high degree of precision (and know what to look for, what to challenge, etc.).
- samxli 6mo ago
  This is also why we need to teach critical thinking skills better in schools so future humans can use AI more effectively.

gobdovan 6mo ago

By the logic that today's news is fundamental to know as true, there really is no point in reading books older than 6 months old. If Einstein woke up from a coma, he'd be useless, as he doesn't even know who won the World Cup. For real now, if an AI can help you solve a problem using 2,000 years of human logic, does it really matter if it's "skewed" away from a political shift that happened three weeks ago? I also don't believe that everybody I know is idiosyncratic in the way they view the world. And even if they were, I'd probably just pay attention to the things that are directly relevant to me. So probably I'll misunderstand most of what they say anyway.

Yokohiii 6mo ago
The post doesn't question the value of historic knowledge. The idea is that a base model could be years old, with all it's faults; superficially fixed and extended with knowledge over time. She points to a research that LLMs don't "fully believe" this new knowledge. The skew could be much longer then a few weeks.

measurablefunc 6mo ago

Calculators endanger the development of mental arithmetic skills as well.

layer8 6mo ago
Indeed, which is why it’s preferable to only start using them after some arithmetic maturity is achieved.
add-sub-mul-div 6mo ago
Right, which is why people make bad money decisions in everyday scenarios. People don't pull out their calculator at the grocery store, but they also never had to get good at doing simple math in their head due to the calculator.
jojomodding 6mo ago
And we collectively decided that it's fine, you don't actually need to be able to solve 1234×5678 in your head. But I am not sure you can compartmentalize the specific skill we can out-source to AI. I would not agree with "you don't need to be able to think in your head."

toooomato 6mo ago

[dead]

svnt 6mo ago

It is a quirky article but the author, instead of engaging with information sources to understand what important thoughts people have had about these topics, feels the best thing to do is introduce new terms that other terms already exist for. This is basically just inductive bias plus the AI homogenization idea producing a distribution shift. This is what happens in thought-isolation. It isn’t better than educating yourself, whether that education involves AI or not. Phillip Kitcher is known for epistemic monoculture, Dawkins and then Henrich popularized collective intelligence and cultural evolution. The thing about these fear pieces is concepts like the hollowed mind are reductive and that reductionism is based on a reductive view of (usually other) people. But what actually happens is we have formalized processes and can externalize them. This is a benefit if you can use your newfound capacity and free time for something better, which I think most people ultimately will.

Forgeties79 6mo ago
>This is a benefit if you can use your newfound capacity and free time for something better, which I think most people ultimately will. I think for a lot of of us the problem is that this is not a given. It’s often promised and rarely occurs, especially in the modern era. Increased productivity usually just means increased demands in the workplace.
- svnt 6mo ago
  Yeah I can’t disagree I think it is really disruptive in a corporate-job-expectation setting where you don’t have the same level of agency.
- le-mark 6mo ago
  Similar to What people were saying about television. They imagined it would bring Shakespeare to the masses, but what ultimately ended up happening is television met people where they are hence we got reality TV. There’s no reason to believe AI won’t be similar.
  - vlunkr 6mo ago

contingencies 6mo ago

Strong disagree. The "AI-Assisted Cognition" phrase is loaded. Would you attempt to, for example, simultaneously modify for available ingredients, number of diners, and time-optimize the prep method for a recipe you've never cooked before if you were following an old-school cookbook? No. You'd have to be a pretty solid chef to try all that on at once. Using AI, you might branch out confidently in to new areas, executing all of these modifications simultaneously, and even adapting the output for a specific audience or language. This toy example shows an important property of AI as decision support systems, which are well studied in the military domain: using these systems, we build confidence to act in unfamiliar domains, thereby extending our reach. From this experience we can learn more. The fact that the learning may then occur through, ie. during or after the experience, rather than beforehand, is secondary. It's still there. The fact we didn't know the language the AI translated to for our chef is totally irrelevant. Sitting comfortably at the effective apex of millions of years of human cognitive and technology development with the entire world's knowledge at our fingertips, every day we can extend confidence in novel domains through AI, and enjoy it. We should be feeling pretty damn "developed". Rote formalism and fixed paths in pedagogy are gone: good riddance. This is the hacker age.

Manuel_D 6mo ago

> In early 2026, the USA prepared to invade Greenland and, therefore, the EU4. Only a few months prior to that it was completely unthinkable that the USA would even think about threatening an invasion of Greenland. As AI base models are stuck in the past, they do not easily accept these events as real and often label them as “hypothetical”, “fake news”, or “impossible”. This also affects new models like Gemini 3 Pro, GLM-5 or GPT-5.3-codex5. Isn't this just inherent to any system that takes some time to update? E.g. if a country moves its capital to a different city, then textbooks, maps, etc. are going to contain incorrect information for a while until updated editions are published. A lot of the complaints about AI are really about the drawbacks of information systems more generally, and the failure modes pointed out are rarely novel. The "Cognitive Inbreeding" effect attributed to AI would also have occurred with Google search would it not? Lots of people type the same question into google and read the top results, instead of searching a more diverse set of information sources. It's interesting that the author mentions web search as a way to ameliorate this, when it seems to me that web search is just as capable of causing cognitive inbreeding.

djrorkrmrk 6mo ago
A few years ago, someone blow up a pipeline in eu. Before thwt some people lied about medical stuff. AI is just current scapegoat.
darepublic 6mo ago
I agree that Google kind of serves this role even before llms. But these days people delegate their reasoning, brainstorming to the computer not just lookup. And beyond our generation are those who would have grown up doing this. Therefore I think concern is justified
Yokohiii 6mo ago
In school we were often required to have a specific edition, or we got notice about a certain thing that changed. People that relied on print knew that it could be wrong. Deliberate changes like a name change, will lead to errors, everybody expects and deals with that. In digital space we expect that to be rather unlikely, at least for major maintained sources. I think the difference is that LLMs are a very complex mix of information and concepts, which can be combined in higher orders. So an underlying wrong fact could be undisclosed and contribute to faulty reasoning. A hard fact like a wrong city name would blow up quickly. A wrong assumption about political dynamics is probably harder to detect, as it is a complex mix of information.

dcre 6mo ago

I've never seen an argument like this that, if true, wouldn't also apply to the cognitive offloading we do by relying on culture, by working with others, or working with the artifacts built by others.

0xBA5ED 6mo ago
Cognitive offloading via culture has many forms and many of them are not sustainable at all.
- dcre 6mo ago
  Sure! I don't mean they're all good. I just mean that it can't be cognitive offloading itself that is the problem, but the particular character of it.

darepublic 6mo ago

The original "person who most of humanity talked to" was, I reckon, google dot com

drusepth 6mo ago

This is absolutely something to potentially be worried about, but one thing I never see highlighted in critiques of AI-assisted cognition is that some elements of physiology may not actually be biologically necessary if they can be fully supplanted by some replacement (in this case, new tools). I can't traverse as much land on foot as my ancestors did (my muscles are weaker, my endurance is less, etc), but I can travel even further than they could by car/plane/etc. Nothing about the nature of evolution implies our current cognitive processing is ideal/sacred and shouldn't ever change.

lexandstuff 6mo ago
> I can't traverse as much land on foot as my ancestors did, but I can travel further by car/plane/etc Which is partially how we found ourselves in the midst of an obesity epidemic.
- ToValueFunfetti 6mo ago
  Only either indirectly or extremely partially. Everybody was driving by the 40s in the US but the obesity epidemic didn't start until the late 70s. Really, obesity is an example of exactly what the GP was talking about- our evolved relationship with food is not inherently good, and it's better for us to change our behaviors than to abandon our advancements and return to the food-scarce world we're adapted to.
  - jplusequalt 6mo ago
    >our evolved relationship with food is not inherently good, and it's better for us to change our behaviors than to abandon our advancements and return to the food-scarce world we're adapted to. So are you arguing we should change our relationship with human intelligence? What does that even mean?
    - ToValueFunfetti 6mo ago
      My intent was to argue that the obesity epidemic is not a supporting analogy for their point. I didn't mean to imply that the problems with it analogize back to AI.

mayankd 6mo ago

The cognitive effects are going to be so divergent. While the avid learners will learn knowledge and skills on the fly exponentially faster, the populace offloading thinking to the AI models will see unprecedented cognitive decline. This is similar to the effect that the internet had on knowledge retention but this time on critical thinking

cyanydeez 6mo ago

Do we think AI is similar to being rich, but without all that cash? I mean, they can basically offload most things to other people to think about.

drivebyhooting 6mo ago

If computers are bicycles for the mind and AI are cars, I wonder what the analogue for the obesity epidemic is.

gdubya 6mo ago
That is a sharp and slightly chilling analogy. If Steve Jobs saw the computer as a tool that amplified human effort (the bicycle), and AI represents a tool that automates that effort entirely (the car), then the "obesity epidemic" of the mind is likely Cognitive Atrophy. - Gemini
- antonvs 6mo ago
  > That is a sharp LLM tell right there. > - Gemini Yes, we already know. I suppose you think posting AI slop in this context is funny. It isn't. Also, no, the observation is not sharp. You're being gaslighted and having your cock fluffed by a machine.
antonvs 6mo ago
The obesity epidemic has much less to do with cars and much more to do with cheapness of food and volume consumed. A typical deli sandwich in the US should be enough to last any normal person three days. Same goes for e.g. ice cream from Shake Shack (random example I know, but one I came across recently). If you buy one of these and eat them in one sitting, the answer to "why am I obese" is simply "you eat way too much."
- cyberax 6mo ago
  Sorry, don't exaggerate please. A Subway sandwhich is about 600 calories. That's about 1/3 of the daily 2000 calories standard. A Shake Shack shake tops out at 1010 calories, a half of the daily norm.
  - antonvs 6mo ago
    I'm not talking about Subway. I specifically said deli sandwich. There's no world in which Subway is a deli. As for Shake Shack, a single shake is half of the daily total calorie intake? Are you listening to yourself right now?

giancarlostoro 6mo ago

I think the best way I can put it is probably; this is the same as if you just cheat off someone else in school, you aren't learning much are you? AI is the same thing. Don't just cheat, use it to learn instead.

anigbrowl 6mo ago

So does talking to uninformed people. The size of the group is inversely correlated with deviation from the mean (of IQ, productivity, or whatever proxy for cognitive capability you care to specify). I'm not sure why this is at the top of the page; it's not that it's wrong, it's just a sequence of truisms.

2OEH8eoCRo0 6mo ago

Economic incentives are forcing us to use tools that make us dumber. What does the future hold?