12 ms·

I have the technical knowledge to know how LLMs work, but I still find it pointless to not anthropomorphize, at least to an extent. The language of "generator

I have the technical knowledge to know how LLMs work, but I still find it pointless to not anthropomorphize, at least to an extent. The language of "generator that stochastically produces the next word" is just not very useful when you're talking about, e.g., an LLM that is answering complex world modeling questions or generating a creative story. It's at the wrong level of abstraction, just as if you were discussing an UI events API and you were talking about zeros and ones, or voltages in transistors. Technically fine but totally useless to reach any conclusion about the high-level system. We need a higher abstraction level to talk about higher level phenomena in LLMs as well, and the problem is that we have no idea what happens internally at those higher abstraction levels. So, considering that LLMs somehow imitate humans (at least in terms of output), anthropomorphization is the best abstraction we have, hence people naturally resort to it when discussing what LLMs can do.

grey-area 1y ago
On the contrary, anthropomorphism IMO is the main problem with narratives around LLMs - people are genuinely talking about them thinking and reasoning when they are doing nothing of that sort (actively encouraged by the companies selling them) and it is completely distorting discussions on their use and perceptions of their utility.
- cmenge 1y ago
  I kinda agree with both of you. It might be a required abstraction, but it's a leaky one. Long before LLMs, I would talk about classes / functions / modules like "it then does this, decides the epsilon is too low, chops it up and adds it to the list". The difference I guess it was only to a technical crowd and nobody would mistake this for anything it wasn't. Everybody know that "it" didn't "decide" anything. With AI being so mainstream and the math being much more elusive than a simple if..then I guess it's just too easy to take this simple speaking convention at face value. EDIT: some clarifications / wording
  - flir 1y ago
    Agreeing with you, this is a "can a submarine swim" problem IMO. We need a new word for what LLMs are doing. Calling it "thinking" is stretching the word to breaking point, but "selecting the next word based on a complex statistical model" doesn't begin to capture what they're capable of. Maybe it's cog-nition (emphasis on the cog).
    - whilenot-dev 1y ago
      "predirence" -> prediction meets inference and it sounds a bit like preference
      - psychoslave 1y ago
        Except -ence is a regular morph, and you would rather suffix it to predict(at)-. And prediction is already an hyponym of inference. Why not just use inference then?
        whilenot-dev 1y ago
        I didn't think of prediction in the statistical sense here, but rather as a prophecy based on a vision, something that is inherently stored in a model without the knowledge of the modelers. I don't want to imply any magic or something supernatural here, it's just the juice that goes off the rails sometimes, and it gets overlooked due to the sheer quantity of the weights. Something like unknown bugs in production, but, because they still just represent a valid number in some computation that wouldn't cause any panic, these few bits can show a useful pattern under the right circumstances. Inference would be the part that is deliberately learned and drawn from conclusions based on the training set, like in the "classic" sense of statistical learning.
    - LeonardoTolstoy 1y ago
      What does a submarine do? Submarine? I suppose you "drive" a submarine which is getting to the idea: submarines don't swim because ultimately they are "driven"? I guess the issue is we don't make up a new word for what submarines do, we just don't use human words. I think the above poster gets a little distracted by suggesting the models are creative which itself is disputed. Perhaps a better term, like above, would be to just use "model". They are models after all. We don't make up a new portmanteau for submarines. They float, or drive, or submarine around. So maybe an LLM doesn't "write" a poem, but instead "models a poem" which maybe indeed take away a little of the sketchy magic and fake humanness they tend to be imbued with.
- Al-Khwarizmi 1y ago
  I think it's worth distinguishing between the use of anthropomorphism as a useful abstraction and the misuse by companies to fuel AI hype. For example, I think "chain of thought" is a good name for what it denotes. It makes the concept easy to understand and discuss, and a non-antropomorphized name would be unnatural and unnecessarily complicate things. This doesn't mean that I support companies insisting that LLMs think just like humans or anything like that. By the way, I would say actually anti-anthropomorphism has been a bigger problem for understanding LLMs than anthropomorphism itself. The main proponents of anti-anthropomorphism (e.g. Bender and the rest of "stochastic parrot" and related paper authors) came up with a lot of predictions about things that LLMs surely couldn't do (on account of just being predictors of the next word, etc.) which turned out to be spectacularly wrong.
  - whilenot-dev 1y ago
    I don't know about others, but I much prefer if some reductionist tries to conclude what's technically feasible and is proven wrong over time, than somebody yelling holistic analogies á la "it's sentient, it's intelligent, it thinks like us humans" for the sole dogmatic reason of being a futurist. Tbh I also think your comparison that puts "UI events -> Bits -> Transistor Voltages" as analogy to "AI thinks -> token de-/encoding + MatMul" is certainly a stretch, as the part about "Bits -> Transistor Voltages" applies to both hierarchies as the foundational layer. "chain of thought" could probably be called "progressive on-track-inference" and nobody would roll an eye.
- amelius 1y ago
  I don't agree. Most LLMs have been trained on human data, so it is best to talk about these models in a human way.
  - 4ndrewl 1y ago
    Even the verb 'trained' is contentious wrt anthropomorphism.
    - amelius 1y ago
      Somewhat true but rodents can also be trained ...
      - 4ndrewl 1y ago
        Rodents aren't functions though?
        FeepingCreature 1y ago
        Every computable system, even stateful systems, can be reformulated as a function. If IO can be functional, I don't see why mice can't.
        psychoslave 1y ago
        Well, that's a strong claim of equivalence between computationable models and realty. The consensual view is rather that no map is matching fully the territory, or said otherwise the territory includes ontological components that exceeds even the most sophisticated map that can be ever built.
        FeepingCreature 1y ago
        I believe the consensus view is that physics is computable.
- fenomas 1y ago
  When I see these debates it's always the other way around - one person speaks colloquially about an LLM's behavior, and then somebody else jumps on them for supposedly believing the model is conscious, just because the speaker said "the model thinks.." or "the model knows.." or whatever. To be honest the impression I've gotten is that some people are just very interested in talking about not anthropomorphizing AI, and less interested in talking about AI behaviors, so they see conversations about the latter as a chance to talk about the former.
  - latexr 1y ago
    Respectfully, that is a reflection of the places you hang out in (like HN) and not the reality of the population. Outside the technical world it gets much worse. There are people who killed themselves because of LLMs, people who are in love with them, people who genuinely believe they have “awakened” their own private ChatGPT instance into AGI and are eschewing the real humans in their lives.
    - fenomas 1y ago
      Naturally I'm aware of those things, but I don't think TFA or GGP were commenting on them so I wasn't either.
    - Xss3 1y ago
      The other day a good friend of mine with mental health issues remarked that "his" chatgpt understands him better than most of his friends and gives him better advice than his therapist. It's going to take a lot to get him out of that mindset and frankly I'm dreading trying to compare and contrast imperfect human behaviour and friendships with a sycophantic AI.
      - bonoboTP 1y ago
        It's surprisingly common on reddit that people talk about "my chatgpt", and they don't always seem like the type who are "in a relationship" with the bot or unlocking the secrets of the cosmos with it, but still they write "my chatgpt" and "your chatgpt". I guess the custom prompt and the available context does customize the model for them in some sense, but I suspect they likely have a wrong mental model of how this customization works. I guess they imagine it as their own little model being stored on file at OpenAI and as they interact with it, it's being shaped by it, and each time they connect, their model is retrieved from the cloud storage and they connect to it or something.
- stoneyhrm1 1y ago
  I thought this too but then began to think about it from the perspective of the programmers trying to make it imitate human learning. That's what a nn is trying to do at the end of the day, and in the same way I train myself by reading problems and solutions, or learning vocab at a young age, it does so by tuning billions of parameters. I think these models do learn similarly. What does it even mean to reason? Your brain knows certain things so it comes to certain conclusions, but it only knows those things because it was ''trained'' on those things. I reason my car will crash if I go 120 mph on the other side of the road because previously I have 'seen' where the input is a car going 120mph has a high probability of producing a crash, and similarly have seen input where the car is going on the other side of the road, producing a crash. Combining the two would tell me it's a high probability.
- losvedir 1y ago
  Well "reasoning" refers to Chain-of-Thought and if you look at the generated prompts it's not hard to see why it's called that. That said, it's fascinating to me that it works (and empirically, it does work; a reasoning model generating tens of thousands of tokens while working out the problem does produce better results). I wish I knew why. A priori I wouldn't have expected it, since there's no new input. That means it's all "in there" in the weights already. I don't see why it couldn't just one shot it without all the reasoning. And maybe the future will bring us more distilled models that can do that, or they can tease out all that reasoning with more generated training data, to move it from dispersed around the weights -> prompt -> more immediately accessible in the weights. But for now "reasoning" works. But then, at the back of my mind is the easy answer: maybe you can't optimize it. Maybe the model has to "reason" to "organize its thoughts" and get the best results. After all, if you give me a complicated problem I'll write down hypotheses and outline approaches and double check results for consistency and all that. But now we're getting dangerously close to the "anthropomorphization" that this article is lamenting.
  - sdenton4 1y ago
    CoT gives the model more time to think and process the inputs it has. To give an extreme example, suppose you are using next token prediction to answer 'Is P==NP?' The tiny number of input tokens means that there's a tiny amount of compute to dedicate to producing an answer. A scratchpad allows us to break free of the short-inputs problem. Meanwhile, things can happen in the latent representation which aren't reflected in the intermediate outputs. You could, instead of using CoT, say "Write a recipe for a vegetarian chile, along with a lengthy biographical story relating to the recipe. Afterwards, I will ask you again about my original question." And the latents can still help model the primary problem, yielding a better answer than you would have gotten with the short input alone. Along these lines, I believe there are chain of thought studies which find that the content of the intermediate outputs don't actually matter all that much...
  - shakadak 1y ago
    > I don't see why it couldn't just one shot it without all the reasoning. That's reminding me of deep neural networks where single layer networks could achieve the same results, but the layer would have to be excessively large. Maybe we're re-using the same kind of improvement, scaling in length instead of width because of our computation limitations ?
- bunderbunder 1y ago
  "All models are wrong, but some models are useful," is the principle I have been using to decide when to go with an anthropomorphic explanation. In other words, no, they never accurately describe what the LLM is actually doing. But sometimes drawing an analogy to human behavior is the most effective way to pump others' intuition about a particular LLM behavior. The trick is making sure that your audience understands that this is just an analogy, and that it has its limitations. And it's not completely wrong. Mimicking human behavior is exactly what they're designed to do. You just need to keep reminding people that it's only doing so in a very superficial and spotty way. There's absolutely no basis for assuming that what's happening on the inside is the same.
  - Veen 1y ago
    Some models are useful in some contexts but wrong enough to be harmful in others.
    - bunderbunder 1y ago
      All models are useful in some contexts but wrong enough to be harmful in others. Relatedly, the alternative to pragmatism is analysis paralysis.
- bakuninsbart 1y ago
  > people are genuinely talking about them thinking and reasoning when they are doing nothing of that sort With such strong wording, it should be rather easy to explain how our thinking differs from what LLMs do. The next step - showing that what LLMs do precludes any kind of sentience is probably much harder.
- ordu 1y ago
  > On the contrary, anthropomorphism IMO is the main problem with narratives around LLMs I hold a deep belief that anthropomorphism is a way the human mind words. If we take for granted the hypothesis of Franz de Waal, that human mind developed its capabilities due to political games, and then think about how it could later lead to solving engineering and technological problems, then the tendency of people to anthropomorphize becomes obvious. Political games need empathy or maybe some other kind of -pathy, that allows politicians to guess motives of others looking at their behaviors. Political games directed the evolution to develop mental instruments to uncover causality by watching at others and interacting with them. Now, to apply these instruments to inanimate world all you need is to anthropomorphize inanimate objects. Of course, it leads sometimes to the invention of gods, or spirits, or other imaginary intelligences behinds things. And sometimes these entities get in the way of revealing the real causes of events. But I believe that to anthropomorphize LLMs (at the current stage of their development) is not just the natural thing for people but a good thing as well. Some behavior of LLMs is easily described in terms of psychology; some cannot be described or at least not so easy. People are seeking ways to do it. Projecting this process into the future, I can imagine how there will be a kind of consensual LLMs "theory" that explains some traits of LLMs in terms of human psychology and fails to explain other traits, so they are explained in some other terms... And then a revolution happens, when a few bright minds come and say that "anthropomorphism is bad, it cannot explain LLM" and they propose something different. I'm sure it will happen at some point in the future, but not right now. And it will happen not like that: not just because someone said that anthropomorphism is bad, but because they proposed another way to talk about reasons behind LLMs behavior. It is like with scientific theories: they do not fail because they become obviously wrong, but because other, better theories replace them. It doesn't mean, that there is no point to fight anthropomorphism right now, but this fight should be directed at searching for new ways to talk about LLMs, not to show at the deficiencies of anthropomorphism. To my mind it makes sense to start not with deficiencies of anthropomorphism but with its successes. What traits of LLMs it allows us to capture, which ideas about LLMs are impossible to wrap into words without thinking of LLMs as of people?
- marviel 1y ago
  how do you account for the success of reasoning models? I agree these things don't think like we do, and that they have weird gaps, but to claim they can't reason at all doesn't feel grounded.
- godelski 1y ago
  Serendipitous name... In part I agree with the parent. >> it pointless to *not* anthropomorphize, at least to an extent. I agree that it is pointless to not anthropomorphize because we are humans and we will automatically do this. Willingly or unwillingly. On the other hand, it generates bias. This bias can lead to errors. So the real answer is (imo) that it is fine to anthropomorphise but recognize that while doing so can provide utility and help us understand, it is WRONG. Recognizing that it is not right and cannot be right provides us with a constant reminder to reevaluate. Use it, but double check, and keep checking making sure you understand the limitations of the analogy. Understanding when and where it applies, where it doesn't, and most importantly, where you don't know if it does or does not. The last is most important because it helps us form hypotheses that are likely to be testable (likely, not always. Also, much easier said than done). So I pick a "grey area". Anthropomorphization is a tool that can be helpful. But like any tool, it isn't universal. There is no "one-size-fits-all" tool. Literally, one of the most important things for any scientist is to become an expert at the tools you use. It's one of the most critical skills of *any expert*. So while I agree with you that we should be careful of anthropomorphization, I disagree that it is useless and can never provide information. But I do agree that quite frequently, the wrong tool is used for the right job. Sometimes, hacking it just isn't good enough.
- UncleOxidant 1y ago
  It's not just distorting discussions it's leading people to put a lot of faith in what LLMs are telling them. Was just on a zoom an hour ago where a guy working on a startup asked ChatGPT about his idea and then emailed us the result for discussion in the meeting. ChatGPT basically just told him what he wanted to hear - essentially that his idea was great and it would be successful ("if you implement it correctly" was doing a lot of work). It was a glowing endorsement of the idea that made the guy think that he must have a million dollar idea. I had to be "that guy" who said that maybe ChatGPT was telling him what he wanted to hear based on the way the question was formulated - tried to be very diplomatic about it and maybe I was a bit too diplomatic because it didn't shake his faith in what ChatGPT had told him.
  - TimTheTinker 1y ago
    LLMs directly exploit a human trust vuln. Our brains tend to engage with them relationally and create an unconscious functional belief that an agent on the other end is responding with their real thoughts, even when we know better. AI apps ought to at minimum warn us that their responses are not anyone's (or anything's) real thoughts. But the illusion is so powerful that many people would ignore the warning.
- nomel 1y ago
  > people are genuinely talking about them thinking and reasoning when they are doing nothing of that sort Do you believe thinking/reasoning is a binary concept? If not, do you think the current top LLM are before or after the 50% mark? What % do you think they're at? What % range do you think humans exhibit?
raincole 1y ago
I've said that before: we have been anthropomorphizing computers since the dawn of information age. - Read and write - Behaviors that separate humans from animals. Now used for input and output. - Server and client - Human social roles. Now used to describe network architecture. - Editor - Human occupation. Now a kind of software. - Computer - Human occupation! And I'm sure people referred their cars and ships as 'her' before the invention of computers.
- latexr 1y ago
  You are conflating anthropomorphism with personification. They are not the same thing. No one believes their guitar or car or boat is alive and sentient when they give it a name or talk to or about it. https://www.masterclass.com/articles/anthropomorphism-vs-personification https://www.masterclass.com/articles/anthropomorphism-vs-per...
  - raincole 1y ago
    But the author used "anthropomorphism" the same way as I did. I guess we both mean "personification" then. > we talk about "behaviors", "ethical constraints", and "harmful actions in pursuit of their goals". All of these are anthropocentric concepts that - in my mind - do not apply to functions or other mathematical objects. One talking about a program's "behaviors", "actions" or "goals" doesn't mean they believe the program is sentient. Only "ethical constraints" is suspiciously anthropomorphizing.
    - latexr 1y ago
      > One talking about a program's "behaviors", "actions" or "goals" doesn't mean they believe the program is sentient. Except that is exactly what we’re seeing with LLMs. People believing exactly that.
      - raincole 1y ago
        Perhaps a few mentally unhinged people do. A bit of anecdote: last year I hung out with a bunch of old classmates that I hadn't seen for quite a while. None of them works in tech. Surprisingly to me, all of them have ChatGPT installed on their phones. And unsurprisingly to me, none of them treated it like an actual intelligence. That makes me wonder where those who think ChatGPT is sentient come from. (It's a bit worrisome that several of them thought it worked "like Google search and Google translation combined", even by the time ChatGPT couldn't do web search...!)
mercer 1y ago
I get the impression after using language models for quite a while that perhaps the one thing that is riskiest to anthropomorphise is the conversational UI that has become the default for many people. A lot of the issues I'd have when 'pretending' to have a conversation are much less so when I either keep things to a single Q/A pairing, or at the very least heavily edit/prune the conversation history. Based on my understanding of LLM's, this seems to make sense even for the models that are trained for conversational interfaces. so, for example, an exchange with multiple messages, where at the end I ask the LLM to double-check the conversation and correct 'hallucinations', is less optimal than something like asking for a thorough summary at the end, and then feeding that into a new prompt/conversation, as the repetition of these falsities, or 'building' on them with subsequent messages, is more likely to make them a stronger 'presence' and as a result perhaps affect the corrections. I haven't tested any of this thoroughly, but at least with code I've definitely noticed how a wrong piece of code can 'infect' the conversation.
- Xss3 1y ago
  This. If an AI spits out incorrect code then i immediately create a new chat and reprompt with additional context. 'Dont use regex for this task' is a common addition for the new chat. Why does AI love regex for simple string operations?
  - naasking 1y ago
    I used to do this as well, but Gemini 2.5 has improved on this quite a bit and I don't find myself needing to do it as much anymore.
endymion-light 1y ago
This is why I actually really love the description of it as a "Shoggoth" - it's more abstract, slightly floaty but it achieves the purpose of not treating and anthropomising it as a human being while not treating LLMs as a collection of predictive words.
tempfile 1y ago
The "point" of not anthropomorphizing is to refrain from judgement until a more solid abstraction appears. The problem with explaining LLMs in terms of human behaviour is that, while we don't clearly understand what the LLM is doing, we understand human cognition even less! There is literally no predictive power in the abstraction "The LLM is thinking like I am thinking". It gives you no mechanism to evaluate what tasks the LLM "should" be able to do. Seriously, try it. Why don't LLMs get frustrated with you if you ask them the same question repeatedly? A human would. Why are LLMs so happy to give contradictory answers, as long as you are very careful not to highlight the contradictory facts? Why do earlier models behave worse on reasoning tasks than later ones? These are features nobody, anywhere understands. So why make the (imo phenomenally large) leap to "well, it's clearly just a brain"? It is like someone inventing the aeroplane and someone looks at it and says "oh, it's flying, I guess it's a bird". It's not a bird!
- CuriousSkeptic 1y ago
  > Why don't LLMs get frustrated with you if you ask them the same question repeatedly? To be fair, I have had a strong sense of Gemini in particular becoming a lot more frustrated with me than GPT or Claude. Yesterday I had it ensuring me that it was doing a great job, it was just me not understanding the challenge but it would break it down step by step just to make it obvious to me (only to repeat the same errors, but still) I’ve just interpreted it as me reacting to the lower amount of sycophancy for now
  - danielbln 1y ago
    In addition, when the boss man asks for the same thing repeatedly then the underling might get frustrated as hell, but they won't be telling that to the boss.
  - jibal 1y ago
    Point out to an LLM that it has no mental states and thus isn't capable of being frustrated (or glad that your program works or hoping that it will, etc. ... I call them out whenever they ascribe emotions to themselves) and they will confirm that ... you can coax from them quite detailed explanations of why and how it's an illusion. Of course they will quickly revert to self-anthropomorphizing language, even after promising that they won't ... because they are just pattern matchers producing the sort of responses that conforms to the training data, not cognitive agents capable of making or keeping promises. It's an illusion.
psychoslave 1y ago
LLM are as far away from your description as ASM is from the underlying architecture. The anthropomorohic abstraction is as nice as any metaphore which fall apart the very moment you put a foot outside what it allows to shallowoly grab. But some people will put far more amount to push force a confortable analogy rather than admit it has some limits and to use the new tool in a more relevant way you have to move away from this confort zone.
adityaathalye 1y ago
My brain refuses to join the rah-rah bandwagon because I cannot see them in my mind’s eye. Sometimes I get jealous of people like GP and OP who clearly seem to have the sight. (Being a serial math exam flunker might have something to do with it. :)))) Anyway, one does what one can. (I've been trying to picture abstract visual and semi-philosophical approximations which I’ll avoid linking here because they seem to fetch bad karma in super-duper LLM enthusiast communities. But you can read them on my blog and email me scathing critiques, if you wish :sweat-smile:.)
woliveirajr 1y ago
I'd take it in reverse order: the problem isn't that it's possible to have a computer that "stochastically produces the next word" and can fool humans, it's why / how / when humans evolved to have technological complexity when the majority (of people) aren't that different from a stochastic process.
pmg101 1y ago
I remember Dawkins talking about the "intentional stance" when discussing genes in The Selfish Gene. It's flat wrong to describe genes as having any agency. However it's a useful and easily understood shorthand to describe them in that way rather than every time use the full formulation of "organisms who tend to possess these genes tend towards these behaviours." Sometimes to help our brains reach a higher level of abstraction, once we understand the low level of abstraction we should stop talking and thinking at that level.
- jibal 1y ago
  The intentional stance was Daniel Dennett's creation and a major part of his life's work. There are actually (exactly) three stances in his model: the physical stance, the design stance, and the intentional stance. https://en.wikipedia.org/wiki/Intentional_stance https://en.wikipedia.org/wiki/Intentional_stance I think the design stance is appropriate for understanding and predicting LLM behavior, and the intentional stance is not.
  - pmg101 1y ago
    Thanks for the correction. I guess both thinkers took a somewhat similar position and I somehow remembered Dawkins's argument but Dennett's term. The term is memorable. Do you want to describe WHY you think the design stance is appropriate here but the intentional stance is not?
    - jibal 1y ago
      Dawkins and Dennett made frequent mention of each other's ideas ... e.g., Dennett discussed "selfish genes" in his book Darwin's Dangerous Idea, and elsewhere. As for your question: the intentional stance allows us to predict the behavior of goal-driven systems in terms of what we would expect a rational agent with those goals to do ... but LLMs are not goal-driven oriented systems, and not only isn't it necessary to treat them as such but doing so leads to erroneous expectations. Here's something that you will never see an LLM do but we see humans do all the time: offer an unsolicited response saying that they realized that they made a mistake, or that they have an improvement on their previous argument ... humans do that because they are driven to get things right or to come out ahead in an argument or just to engage in conversation for various reasons... LLMs have no drives, they just match text. An LLM may be able to do great on an LSAT, but it won't pursue a career in law ... it just isn't that sort of thing. Here's what Claude said about it (I think it got it right): The Intentional Stance Overstates LLM Agency Dennett's intentional stance involves treating systems as rational agents with beliefs, desires, and intentions when this proves useful for prediction and explanation. While this might seem to fit LLMs—we naturally say things like "GPT-4 believes X" or "Claude wants to be helpful"—this framing is misleading for several reasons. LLMs lack the continuity of goals and persistent world-models that characterize genuine intentional systems. When an LLM generates a response, it's not pursuing long-term objectives or maintaining beliefs across conversations. Each response emerges from pattern matching against training data, not from reasoning about how to achieve desired outcomes. The "beliefs" we might attribute to an LLM are actually statistical regularities in text, not representations the system uses to guide action. The Design Stance Captures What's Actually Happening The design stance, by contrast, explains behavior by reference to what a system was designed to do and how it was built to function. This fits LLMs perfectly: they were designed to predict and generate text that resembles human language patterns. Their behavior emerges from their training objective (next-token prediction), their architecture (transformer networks), and their training data. When we adopt the design stance toward LLMs, we correctly understand that their seemingly intelligent responses result from sophisticated pattern recognition and statistical inference over vast text corpora. They're not reasoning about beliefs or pursuing goals—they're executing the functions they were designed to perform. This explains both their capabilities (remarkable fluency in text generation) and their limitations (inconsistency, hallucination, lack of genuine understanding). Why This Matters The design stance helps us maintain appropriate expectations about what LLMs can and cannot do, while the intentional stance risks anthropomorphizing systems that, however sophisticated, remain fundamentally different from minds with genuine intentionality.
lo_zamoyski 1y ago
These anthropomorphizations are best described as metaphors when used by people to describe LLMs in common or loose speech. We already use anthropomorphic metaphors when talking about computers. LLMs, like all computation, are a matter of simulation; LLMs can appear to be conversing without actually conversing. What distinguishes the real thing from the simulation is the cause of the appearance of an effect. Problems occur when people forget these words are being used metaphorically, as if they were univocal. Of course, LLMs are multimodal and used to simulate all sorts of things, not just conversation. So there are many possible metaphors we can use, and these metaphors don't necessarily align with the abstractions you might use to talk about LLMs accurately. This is like the difference between "synthesizes text" (abstraction) and "speaks" (metaphor), or "synthesizes images" (abstraction) and "paints" (metaphor). You can use "speaks" or "paints" to talk about the abstractions, of course.
overfeed 1y ago
> We need a higher abstraction level to talk about higher level phenomena in LLMs as well, and the problem is that we have no idea what happens internally at those higher abstraction levels We do know what happens at higher abstraction levels; the design of efficient networks, and the steady beat of SOTA improvements all depend on understanding how LLMs work internally: choice of network dimensions, feature extraction, attention, attention heads, caching, the peculiarities of high-dimensions and avoiding overfitting are all well-understood by practitioners. Anthropomorphization is only necessary in pop-science articles that use a limited vocabulary. IMO, there is very little mystery, but lots of deliberate mysticism, especially about future LLMs - the usual hype-cycle extrapolation.
lawlessone 1y ago
One thing i find i keep forgetting is that asking an LLM why it makes a particular decision is almost pointless. It's reply isn't actually going to be why i did a thing. It's reply is going to be whatever is the most probably string of words that fit as a reason.
amdivia 1y ago
I beg to differ. Anthropomorphizing might blind us to solutions to existing problems. Perhaps instead of trying to come up with the correct prompt for a LLM, there exists a string of words (not necessary ones that make sense) that will get the LLM to a better position to answer given questions. When we anthropomorphize we are inherently ignore certain parts of how LLMs work, and imagining parts that don't even exist
- meroes 1y ago
  > there exists a string of words (not necessary ones that make sense) that will get the LLM to a better position to answer exactly. The opposite is also true. You might supply more clarifying information to the LLM, which would help any human answer, but it actually degrades the LLM's output.
  - mvieira38 1y ago
    This is frequently the case IME, especially with chat interfaces. One or two bad messages and you derail the quality
    - lawlessone 1y ago
      You can just throw in words to bias it towards certain outcomes too. Same applies with image generators or course.
aaroninsf 1y ago
That higher level does exist, indeed a lot philosophy of mind then cognitive science has been investigating exactly this space and devising contested professional nomenclature and modeling about such things for decades now. A useful anchor concept is that of world model, which is what "learning Othello" and similar work seeks to tease out. As someone who worked in precisely these areas for years and has never stopped thinking about them, I find it at turns perplexing, sigh-inducing, and enraging, that the "token prediction" trope gained currency and moreover that it continues to influence people's reasoning about contemporary LLM, often as subtext: an unarticulated fundamental model, which is fundamentally wrong in its critical aspects. It's not that this description of LLM is technically incorrect; it's that it is profoundly _misleading_ and I'm old enough and cynical enough to know full well that many of those who have amplified it and continue to do so, know this very well indeed. Just as the lay person fundamentally misunderstands the relationship between "programming" and these models, and uses slack language in argumentation, the problem with this trope and the reasoning it entails is that what is unique and interesting and valuable about LLM for many applications and interests is how they do what they do. At that level of analysis there is a very real argument to be made that the animal brain is also nothing more than an "engine of prediction," whether the "token" is a byte stream or neural encoding is quite important but not nearly important as the mechanics of the system which operates on those tokens. To be direct, it is quite obvious that LLM have not only vestigial world models, but also self-models; and a general paradigm shift will come around this when multimodal models are the norm: because those systems will share with we animals what philosophers call phenomenology, a model of things as they are "perceived" through the senses. And like we humans, these perceptual models (terminology varies by philosopher and school...) will be bound to the linguistic tokens (both heard and spoken, and written) we attach to them. Vestigial is a key word but an important one. It's not that contemporary LLM have human-tier minds, nor that they have animal-tier world modeling: but they can only "do what they do" because they have such a thing. Of looming importance—something all of us here should set aside time to think about—is that for most reasonable contemporary theories of mind, a self-model embedded in a world-model, with phenomenology and agency, is the recipe for "self" and self-awareness. One of the uncomfortable realities of contemporary LLM already having some vestigial self-model, is that while they are obviously not sentient, nor self-aware, as we are, or even animals are, it is just as obvious (to me at least) that they are self-aware in some emerging sense and will only continue to become more so. Among the lines of finding/research most provocative in this area is the ongoing often sensationalized accounting in system cards and other reporting around two specific things about contemporary models: - they demonstrate behavior pursuing self-preservation - they demonstrate awareness of when they are being tested We don't—collectively or individually—yet know what these things entail, but taken with the assertion that these models are developing emergent self-awareness (I would say: necessarily and inevitably), we are facing some very serious ethical questions. The language adopted by those capitalizing and capitalizing _from_ these systems so far is IMO of deep concern, as it betrays not just disinterest in our civilization collectively benefiting from this technology, but also, that the disregard for human wellbeing implicit in e.g. the hostility to UBI, or, Altman somehow not seeing a moral imperative to remain distant from the current adminstation, implies directly a much greater disregard for "AI wellbeing." That that concept is today still speculative is little comfort. Those of us watching this space know well how fast things are going, and don't mistake plateaus for the end of the curve. I do recommend taking a step back from the line-level grind to give these things some thought. They are going to shape the world we live out our days in and our descendents will spend all of theirs in.
jll29 1y ago
The details in how I talk about LLMs matter. If I use human-related terminology as a shortcut, as some kind of macro to talk at a higher level/more efficiently about something I want to do that might be okay. What is not okay is talking in a way that implies intent, for example. Compare: "The AI doesn't want to do that." versus "The model doesn't do that with this prompt and all others we tried." The latter way of talking is still high-level enough but avoids equating/confusing the name of a field with a sentient being. Whenever I hear people saying "an AI" I suggest they replace AI with "statistics" to make it obvious how problematic anthropomorphisms may have become: *"The statistics doesn't want to do that."
- dmitsuki 1y ago
  The only reason that sounds weird to you is because you have the experience of being human. Human behavior is not magic. It's still just statistics. You go to the bathroom when you have to pee not because some magical concept of consciousness, but because a reciptor in your brain goes off and starts the chain of making you go to the bathroom. AI's are not magic, but nobody has sufficiently provided any proof we are somehow special either.
TeMPOraL 1y ago
Agreed. I'm also in favor of anthropomorphizing, because not doing so confuses people about the nature and capabilities of these models even more. Whether it's hallucinations, prompt injections, various other security vulnerabilities/scenarios, or problems with doing math, backtracking, getting confused - there's a steady supply of "problems" that some people are surprised to discover and even more surprised this isn't being definitively fixed. Thing is, none of that is surprising, and these things are not bugs, they're flip side of the features - but to see that, one has to realize that humans demonstrate those exact same failure modes. Especially when it comes to designing larger systems incorporating LLM "agents", it really helps to think of them as humans - because the problems those systems face are exactly the same as you get with systems incorporating people, and mostly for the same underlying reasons. Anthropomorphizing LLMs cuts through a lot of misconceptions and false paths, and helps one realize that we have millennia of experience with people-centric computing systems (aka. bureaucracy) that's directly transferrable.
- godelski 1y ago
  I disagree. Anthropomorphization can be a very useful tool but I think it is currently over used and is a very tricky tool to use when communicating with a more general audience. I think looking at physics might be a good example. We love our simplified examples and there's a big culture of trying to explain things to the lay person (mostly because the topics are incredibly complex). But how many people have misunderstood an observer of a quantum event with "a human" and do not consider "a photon" as an observer? How many people think in Schrodinger's Cat that the cat is both alive and dead?[0] Or believe in a multiverse. There's plenty of examples we can point to. While these analogies *can* be extremely helpful, they *can* also be extremely harmful. This is especially true as information is usually passed through a game of telephone[1]. There is information loss and with it, interpretation becomes more difficult. Often a very subtle part can make a critical distinction. I'm not against anthropomorphization[2], but I do think we should be cautious about how we use it. The imprecise nature of it is the exact reason we should be mindful of when and how to use it. We know that the anthropomorphized analogy is wrong. So we have to think about "how wrong" it is for a given setting. We should also be careful to think about how it may be misinterpreted. That's all I'm trying to say. And isn't this what we should be doing if we want to communicate effectively? [0] It is not. It is either. The point of this thought experiment is that we cannot know the answer without looking inside. There is information loss and the event is not deterministic. It directly relates to the Heisenberg Uncertainty Principle, Godel's Incompleteness, or the Halting Problem. All these things are (loosely) related around the inability to have absolute determinism. [1] https://en.wikipedia.org/wiki/Telephone_game https://en.wikipedia.org/wiki/Telephone_game [2] https://news.ycombinator.com/item?id=44494022 https://news.ycombinator.com/item?id=44494022
seanhunter 1y ago
Exactly. We use anthropomorphic language absolutely all the time when describing different processes for this exact reason - it is a helpful abstraction that allows us to easily describe what’s going on at a high level. “My headphones think they’re connected, but the computer can’t see them”. “The printer thinks it’s out of paper, but it’s not”. “The optimisation function is trying to go down nabla f”. “The parking sensor on the car keeps going off because it’s afraid it’s too close to the wall”. “The client is blocked, because it still needs to get a final message from the server”. …and one final one which I promise you is real because I overheard it “I’m trying to airdrop a photo, but our phones won’t have sex”.
lowsong 1y ago
> The language of "generator that stochastically produces the next word" is just not very useful when you're talking about, e.g., an LLM that is answering complex world modeling questions or generating a creative story. But it isn't modelling. It's been shown time, and time, and time again that LLMs have no internal "model" or "view". This is exactly and precisely why you should not anthropomorphize. And again, the output of an LLM is, by definition, not "creative". Your saying we should anthropomorphize these models when the examples you give are already doing that.

grey-area 1y ago

On the contrary, anthropomorphism IMO is the main problem with narratives around LLMs - people are genuinely talking about them thinking and reasoning when they are doing nothing of that sort (actively encouraged by the companies selling them) and it is completely distorting discussions on their use and perceptions of their utility.

cmenge 1y ago
I kinda agree with both of you. It might be a required abstraction, but it's a leaky one. Long before LLMs, I would talk about classes / functions / modules like "it then does this, decides the epsilon is too low, chops it up and adds it to the list". The difference I guess it was only to a technical crowd and nobody would mistake this for anything it wasn't. Everybody know that "it" didn't "decide" anything. With AI being so mainstream and the math being much more elusive than a simple if..then I guess it's just too easy to take this simple speaking convention at face value. EDIT: some clarifications / wording
- flir 1y ago
  Agreeing with you, this is a "can a submarine swim" problem IMO. We need a new word for what LLMs are doing. Calling it "thinking" is stretching the word to breaking point, but "selecting the next word based on a complex statistical model" doesn't begin to capture what they're capable of. Maybe it's cog-nition (emphasis on the cog).
  - whilenot-dev 1y ago
    "predirence" -> prediction meets inference and it sounds a bit like preference
    - psychoslave 1y ago
      Except -ence is a regular morph, and you would rather suffix it to predict(at)-. And prediction is already an hyponym of inference. Why not just use inference then?
      - whilenot-dev 1y ago
        I didn't think of prediction in the statistical sense here, but rather as a prophecy based on a vision, something that is inherently stored in a model without the knowledge of the modelers. I don't want to imply any magic or something supernatural here, it's just the juice that goes off the rails sometimes, and it gets overlooked due to the sheer quantity of the weights. Something like unknown bugs in production, but, because they still just represent a valid number in some computation that wouldn't cause any panic, these few bits can show a useful pattern under the right circumstances. Inference would be the part that is deliberately learned and drawn from conclusions based on the training set, like in the "classic" sense of statistical learning.
  - LeonardoTolstoy 1y ago
    What does a submarine do? Submarine? I suppose you "drive" a submarine which is getting to the idea: submarines don't swim because ultimately they are "driven"? I guess the issue is we don't make up a new word for what submarines do, we just don't use human words. I think the above poster gets a little distracted by suggesting the models are creative which itself is disputed. Perhaps a better term, like above, would be to just use "model". They are models after all. We don't make up a new portmanteau for submarines. They float, or drive, or submarine around. So maybe an LLM doesn't "write" a poem, but instead "models a poem" which maybe indeed take away a little of the sketchy magic and fake humanness they tend to be imbued with.
Al-Khwarizmi 1y ago
I think it's worth distinguishing between the use of anthropomorphism as a useful abstraction and the misuse by companies to fuel AI hype. For example, I think "chain of thought" is a good name for what it denotes. It makes the concept easy to understand and discuss, and a non-antropomorphized name would be unnatural and unnecessarily complicate things. This doesn't mean that I support companies insisting that LLMs think just like humans or anything like that. By the way, I would say actually anti-anthropomorphism has been a bigger problem for understanding LLMs than anthropomorphism itself. The main proponents of anti-anthropomorphism (e.g. Bender and the rest of "stochastic parrot" and related paper authors) came up with a lot of predictions about things that LLMs surely couldn't do (on account of just being predictors of the next word, etc.) which turned out to be spectacularly wrong.
- whilenot-dev 1y ago
  I don't know about others, but I much prefer if some reductionist tries to conclude what's technically feasible and is proven wrong over time, than somebody yelling holistic analogies á la "it's sentient, it's intelligent, it thinks like us humans" for the sole dogmatic reason of being a futurist. Tbh I also think your comparison that puts "UI events -> Bits -> Transistor Voltages" as analogy to "AI thinks -> token de-/encoding + MatMul" is certainly a stretch, as the part about "Bits -> Transistor Voltages" applies to both hierarchies as the foundational layer. "chain of thought" could probably be called "progressive on-track-inference" and nobody would roll an eye.
amelius 1y ago
I don't agree. Most LLMs have been trained on human data, so it is best to talk about these models in a human way.
- 4ndrewl 1y ago
  Even the verb 'trained' is contentious wrt anthropomorphism.
  - amelius 1y ago
    Somewhat true but rodents can also be trained ...
    - 4ndrewl 1y ago
      Rodents aren't functions though?
      - FeepingCreature 1y ago
        Every computable system, even stateful systems, can be reformulated as a function. If IO can be functional, I don't see why mice can't.
        psychoslave 1y ago
        Well, that's a strong claim of equivalence between computationable models and realty. The consensual view is rather that no map is matching fully the territory, or said otherwise the territory includes ontological components that exceeds even the most sophisticated map that can be ever built.
        FeepingCreature 1y ago
        I believe the consensus view is that physics is computable.
fenomas 1y ago
When I see these debates it's always the other way around - one person speaks colloquially about an LLM's behavior, and then somebody else jumps on them for supposedly believing the model is conscious, just because the speaker said "the model thinks.." or "the model knows.." or whatever. To be honest the impression I've gotten is that some people are just very interested in talking about not anthropomorphizing AI, and less interested in talking about AI behaviors, so they see conversations about the latter as a chance to talk about the former.
- latexr 1y ago
  Respectfully, that is a reflection of the places you hang out in (like HN) and not the reality of the population. Outside the technical world it gets much worse. There are people who killed themselves because of LLMs, people who are in love with them, people who genuinely believe they have “awakened” their own private ChatGPT instance into AGI and are eschewing the real humans in their lives.
  - fenomas 1y ago
    Naturally I'm aware of those things, but I don't think TFA or GGP were commenting on them so I wasn't either.
  - Xss3 1y ago
    The other day a good friend of mine with mental health issues remarked that "his" chatgpt understands him better than most of his friends and gives him better advice than his therapist. It's going to take a lot to get him out of that mindset and frankly I'm dreading trying to compare and contrast imperfect human behaviour and friendships with a sycophantic AI.
    - bonoboTP 1y ago
      It's surprisingly common on reddit that people talk about "my chatgpt", and they don't always seem like the type who are "in a relationship" with the bot or unlocking the secrets of the cosmos with it, but still they write "my chatgpt" and "your chatgpt". I guess the custom prompt and the available context does customize the model for them in some sense, but I suspect they likely have a wrong mental model of how this customization works. I guess they imagine it as their own little model being stored on file at OpenAI and as they interact with it, it's being shaped by it, and each time they connect, their model is retrieved from the cloud storage and they connect to it or something.
stoneyhrm1 1y ago
I thought this too but then began to think about it from the perspective of the programmers trying to make it imitate human learning. That's what a nn is trying to do at the end of the day, and in the same way I train myself by reading problems and solutions, or learning vocab at a young age, it does so by tuning billions of parameters. I think these models do learn similarly. What does it even mean to reason? Your brain knows certain things so it comes to certain conclusions, but it only knows those things because it was ''trained'' on those things. I reason my car will crash if I go 120 mph on the other side of the road because previously I have 'seen' where the input is a car going 120mph has a high probability of producing a crash, and similarly have seen input where the car is going on the other side of the road, producing a crash. Combining the two would tell me it's a high probability.
losvedir 1y ago
Well "reasoning" refers to Chain-of-Thought and if you look at the generated prompts it's not hard to see why it's called that. That said, it's fascinating to me that it works (and empirically, it does work; a reasoning model generating tens of thousands of tokens while working out the problem does produce better results). I wish I knew why. A priori I wouldn't have expected it, since there's no new input. That means it's all "in there" in the weights already. I don't see why it couldn't just one shot it without all the reasoning. And maybe the future will bring us more distilled models that can do that, or they can tease out all that reasoning with more generated training data, to move it from dispersed around the weights -> prompt -> more immediately accessible in the weights. But for now "reasoning" works. But then, at the back of my mind is the easy answer: maybe you can't optimize it. Maybe the model has to "reason" to "organize its thoughts" and get the best results. After all, if you give me a complicated problem I'll write down hypotheses and outline approaches and double check results for consistency and all that. But now we're getting dangerously close to the "anthropomorphization" that this article is lamenting.
- sdenton4 1y ago
  CoT gives the model more time to think and process the inputs it has. To give an extreme example, suppose you are using next token prediction to answer 'Is P==NP?' The tiny number of input tokens means that there's a tiny amount of compute to dedicate to producing an answer. A scratchpad allows us to break free of the short-inputs problem. Meanwhile, things can happen in the latent representation which aren't reflected in the intermediate outputs. You could, instead of using CoT, say "Write a recipe for a vegetarian chile, along with a lengthy biographical story relating to the recipe. Afterwards, I will ask you again about my original question." And the latents can still help model the primary problem, yielding a better answer than you would have gotten with the short input alone. Along these lines, I believe there are chain of thought studies which find that the content of the intermediate outputs don't actually matter all that much...
- shakadak 1y ago
  > I don't see why it couldn't just one shot it without all the reasoning. That's reminding me of deep neural networks where single layer networks could achieve the same results, but the layer would have to be excessively large. Maybe we're re-using the same kind of improvement, scaling in length instead of width because of our computation limitations ?
bunderbunder 1y ago
"All models are wrong, but some models are useful," is the principle I have been using to decide when to go with an anthropomorphic explanation. In other words, no, they never accurately describe what the LLM is actually doing. But sometimes drawing an analogy to human behavior is the most effective way to pump others' intuition about a particular LLM behavior. The trick is making sure that your audience understands that this is just an analogy, and that it has its limitations. And it's not completely wrong. Mimicking human behavior is exactly what they're designed to do. You just need to keep reminding people that it's only doing so in a very superficial and spotty way. There's absolutely no basis for assuming that what's happening on the inside is the same.
- Veen 1y ago
  Some models are useful in some contexts but wrong enough to be harmful in others.
  - bunderbunder 1y ago
    All models are useful in some contexts but wrong enough to be harmful in others. Relatedly, the alternative to pragmatism is analysis paralysis.
bakuninsbart 1y ago
> people are genuinely talking about them thinking and reasoning when they are doing nothing of that sort With such strong wording, it should be rather easy to explain how our thinking differs from what LLMs do. The next step - showing that what LLMs do precludes any kind of sentience is probably much harder.
ordu 1y ago
> On the contrary, anthropomorphism IMO is the main problem with narratives around LLMs I hold a deep belief that anthropomorphism is a way the human mind words. If we take for granted the hypothesis of Franz de Waal, that human mind developed its capabilities due to political games, and then think about how it could later lead to solving engineering and technological problems, then the tendency of people to anthropomorphize becomes obvious. Political games need empathy or maybe some other kind of -pathy, that allows politicians to guess motives of others looking at their behaviors. Political games directed the evolution to develop mental instruments to uncover causality by watching at others and interacting with them. Now, to apply these instruments to inanimate world all you need is to anthropomorphize inanimate objects. Of course, it leads sometimes to the invention of gods, or spirits, or other imaginary intelligences behinds things. And sometimes these entities get in the way of revealing the real causes of events. But I believe that to anthropomorphize LLMs (at the current stage of their development) is not just the natural thing for people but a good thing as well. Some behavior of LLMs is easily described in terms of psychology; some cannot be described or at least not so easy. People are seeking ways to do it. Projecting this process into the future, I can imagine how there will be a kind of consensual LLMs "theory" that explains some traits of LLMs in terms of human psychology and fails to explain other traits, so they are explained in some other terms... And then a revolution happens, when a few bright minds come and say that "anthropomorphism is bad, it cannot explain LLM" and they propose something different. I'm sure it will happen at some point in the future, but not right now. And it will happen not like that: not just because someone said that anthropomorphism is bad, but because they proposed another way to talk about reasons behind LLMs behavior. It is like with scientific theories: they do not fail because they become obviously wrong, but because other, better theories replace them. It doesn't mean, that there is no point to fight anthropomorphism right now, but this fight should be directed at searching for new ways to talk about LLMs, not to show at the deficiencies of anthropomorphism. To my mind it makes sense to start not with deficiencies of anthropomorphism but with its successes. What traits of LLMs it allows us to capture, which ideas about LLMs are impossible to wrap into words without thinking of LLMs as of people?
marviel 1y ago
how do you account for the success of reasoning models? I agree these things don't think like we do, and that they have weird gaps, but to claim they can't reason at all doesn't feel grounded.
godelski 1y ago
Serendipitous name... In part I agree with the parent. >> it pointless to *not* anthropomorphize, at least to an extent. I agree that it is pointless to not anthropomorphize because we are humans and we will automatically do this. Willingly or unwillingly. On the other hand, it generates bias. This bias can lead to errors. So the real answer is (imo) that it is fine to anthropomorphise but recognize that while doing so can provide utility and help us understand, it is WRONG. Recognizing that it is not right and cannot be right provides us with a constant reminder to reevaluate. Use it, but double check, and keep checking making sure you understand the limitations of the analogy. Understanding when and where it applies, where it doesn't, and most importantly, where you don't know if it does or does not. The last is most important because it helps us form hypotheses that are likely to be testable (likely, not always. Also, much easier said than done). So I pick a "grey area". Anthropomorphization is a tool that can be helpful. But like any tool, it isn't universal. There is no "one-size-fits-all" tool. Literally, one of the most important things for any scientist is to become an expert at the tools you use. It's one of the most critical skills of *any expert*. So while I agree with you that we should be careful of anthropomorphization, I disagree that it is useless and can never provide information. But I do agree that quite frequently, the wrong tool is used for the right job. Sometimes, hacking it just isn't good enough.
UncleOxidant 1y ago
It's not just distorting discussions it's leading people to put a lot of faith in what LLMs are telling them. Was just on a zoom an hour ago where a guy working on a startup asked ChatGPT about his idea and then emailed us the result for discussion in the meeting. ChatGPT basically just told him what he wanted to hear - essentially that his idea was great and it would be successful ("if you implement it correctly" was doing a lot of work). It was a glowing endorsement of the idea that made the guy think that he must have a million dollar idea. I had to be "that guy" who said that maybe ChatGPT was telling him what he wanted to hear based on the way the question was formulated - tried to be very diplomatic about it and maybe I was a bit too diplomatic because it didn't shake his faith in what ChatGPT had told him.
- TimTheTinker 1y ago
  LLMs directly exploit a human trust vuln. Our brains tend to engage with them relationally and create an unconscious functional belief that an agent on the other end is responding with their real thoughts, even when we know better. AI apps ought to at minimum warn us that their responses are not anyone's (or anything's) real thoughts. But the illusion is so powerful that many people would ignore the warning.
nomel 1y ago
> people are genuinely talking about them thinking and reasoning when they are doing nothing of that sort Do you believe thinking/reasoning is a binary concept? If not, do you think the current top LLM are before or after the 50% mark? What % do you think they're at? What % range do you think humans exhibit?

raincole 1y ago

I've said that before: we have been anthropomorphizing computers since the dawn of information age. - Read and write - Behaviors that separate humans from animals. Now used for input and output. - Server and client - Human social roles. Now used to describe network architecture. - Editor - Human occupation. Now a kind of software. - Computer - Human occupation! And I'm sure people referred their cars and ships as 'her' before the invention of computers.

latexr 1y ago
You are conflating anthropomorphism with personification. They are not the same thing. No one believes their guitar or car or boat is alive and sentient when they give it a name or talk to or about it. https://www.masterclass.com/articles/anthropomorphism-vs-personification https://www.masterclass.com/articles/anthropomorphism-vs-per...
- raincole 1y ago
  But the author used "anthropomorphism" the same way as I did. I guess we both mean "personification" then. > we talk about "behaviors", "ethical constraints", and "harmful actions in pursuit of their goals". All of these are anthropocentric concepts that - in my mind - do not apply to functions or other mathematical objects. One talking about a program's "behaviors", "actions" or "goals" doesn't mean they believe the program is sentient. Only "ethical constraints" is suspiciously anthropomorphizing.
  - latexr 1y ago
    > One talking about a program's "behaviors", "actions" or "goals" doesn't mean they believe the program is sentient. Except that is exactly what we’re seeing with LLMs. People believing exactly that.
    - raincole 1y ago
      Perhaps a few mentally unhinged people do. A bit of anecdote: last year I hung out with a bunch of old classmates that I hadn't seen for quite a while. None of them works in tech. Surprisingly to me, all of them have ChatGPT installed on their phones. And unsurprisingly to me, none of them treated it like an actual intelligence. That makes me wonder where those who think ChatGPT is sentient come from. (It's a bit worrisome that several of them thought it worked "like Google search and Google translation combined", even by the time ChatGPT couldn't do web search...!)

mercer 1y ago

I get the impression after using language models for quite a while that perhaps the one thing that is riskiest to anthropomorphise is the conversational UI that has become the default for many people. A lot of the issues I'd have when 'pretending' to have a conversation are much less so when I either keep things to a single Q/A pairing, or at the very least heavily edit/prune the conversation history. Based on my understanding of LLM's, this seems to make sense even for the models that are trained for conversational interfaces. so, for example, an exchange with multiple messages, where at the end I ask the LLM to double-check the conversation and correct 'hallucinations', is less optimal than something like asking for a thorough summary at the end, and then feeding that into a new prompt/conversation, as the repetition of these falsities, or 'building' on them with subsequent messages, is more likely to make them a stronger 'presence' and as a result perhaps affect the corrections. I haven't tested any of this thoroughly, but at least with code I've definitely noticed how a wrong piece of code can 'infect' the conversation.

Xss3 1y ago
This. If an AI spits out incorrect code then i immediately create a new chat and reprompt with additional context. 'Dont use regex for this task' is a common addition for the new chat. Why does AI love regex for simple string operations?
- naasking 1y ago
  I used to do this as well, but Gemini 2.5 has improved on this quite a bit and I don't find myself needing to do it as much anymore.

endymion-light 1y ago

This is why I actually really love the description of it as a "Shoggoth" - it's more abstract, slightly floaty but it achieves the purpose of not treating and anthropomising it as a human being while not treating LLMs as a collection of predictive words.

tempfile 1y ago

The "point" of not anthropomorphizing is to refrain from judgement until a more solid abstraction appears. The problem with explaining LLMs in terms of human behaviour is that, while we don't clearly understand what the LLM is doing, we understand human cognition even less! There is literally no predictive power in the abstraction "The LLM is thinking like I am thinking". It gives you no mechanism to evaluate what tasks the LLM "should" be able to do. Seriously, try it. Why don't LLMs get frustrated with you if you ask them the same question repeatedly? A human would. Why are LLMs so happy to give contradictory answers, as long as you are very careful not to highlight the contradictory facts? Why do earlier models behave worse on reasoning tasks than later ones? These are features nobody, anywhere understands. So why make the (imo phenomenally large) leap to "well, it's clearly just a brain"? It is like someone inventing the aeroplane and someone looks at it and says "oh, it's flying, I guess it's a bird". It's not a bird!

CuriousSkeptic 1y ago
> Why don't LLMs get frustrated with you if you ask them the same question repeatedly? To be fair, I have had a strong sense of Gemini in particular becoming a lot more frustrated with me than GPT or Claude. Yesterday I had it ensuring me that it was doing a great job, it was just me not understanding the challenge but it would break it down step by step just to make it obvious to me (only to repeat the same errors, but still) I’ve just interpreted it as me reacting to the lower amount of sycophancy for now
- danielbln 1y ago
  In addition, when the boss man asks for the same thing repeatedly then the underling might get frustrated as hell, but they won't be telling that to the boss.
- jibal 1y ago
  Point out to an LLM that it has no mental states and thus isn't capable of being frustrated (or glad that your program works or hoping that it will, etc. ... I call them out whenever they ascribe emotions to themselves) and they will confirm that ... you can coax from them quite detailed explanations of why and how it's an illusion. Of course they will quickly revert to self-anthropomorphizing language, even after promising that they won't ... because they are just pattern matchers producing the sort of responses that conforms to the training data, not cognitive agents capable of making or keeping promises. It's an illusion.

psychoslave 1y ago

LLM are as far away from your description as ASM is from the underlying architecture. The anthropomorohic abstraction is as nice as any metaphore which fall apart the very moment you put a foot outside what it allows to shallowoly grab. But some people will put far more amount to push force a confortable analogy rather than admit it has some limits and to use the new tool in a more relevant way you have to move away from this confort zone.

adityaathalye 1y ago

My brain refuses to join the rah-rah bandwagon because I cannot see them in my mind’s eye. Sometimes I get jealous of people like GP and OP who clearly seem to have the sight. (Being a serial math exam flunker might have something to do with it. :)))) Anyway, one does what one can. (I've been trying to picture abstract visual and semi-philosophical approximations which I’ll avoid linking here because they seem to fetch bad karma in super-duper LLM enthusiast communities. But you can read them on my blog and email me scathing critiques, if you wish :sweat-smile:.)

woliveirajr 1y ago

I'd take it in reverse order: the problem isn't that it's possible to have a computer that "stochastically produces the next word" and can fool humans, it's why / how / when humans evolved to have technological complexity when the majority (of people) aren't that different from a stochastic process.

pmg101 1y ago

I remember Dawkins talking about the "intentional stance" when discussing genes in The Selfish Gene. It's flat wrong to describe genes as having any agency. However it's a useful and easily understood shorthand to describe them in that way rather than every time use the full formulation of "organisms who tend to possess these genes tend towards these behaviours." Sometimes to help our brains reach a higher level of abstraction, once we understand the low level of abstraction we should stop talking and thinking at that level.

jibal 1y ago
The intentional stance was Daniel Dennett's creation and a major part of his life's work. There are actually (exactly) three stances in his model: the physical stance, the design stance, and the intentional stance. https://en.wikipedia.org/wiki/Intentional_stance https://en.wikipedia.org/wiki/Intentional_stance I think the design stance is appropriate for understanding and predicting LLM behavior, and the intentional stance is not.
- pmg101 1y ago
  Thanks for the correction. I guess both thinkers took a somewhat similar position and I somehow remembered Dawkins's argument but Dennett's term. The term is memorable. Do you want to describe WHY you think the design stance is appropriate here but the intentional stance is not?
  - jibal 1y ago
    Dawkins and Dennett made frequent mention of each other's ideas ... e.g., Dennett discussed "selfish genes" in his book Darwin's Dangerous Idea, and elsewhere. As for your question: the intentional stance allows us to predict the behavior of goal-driven systems in terms of what we would expect a rational agent with those goals to do ... but LLMs are not goal-driven oriented systems, and not only isn't it necessary to treat them as such but doing so leads to erroneous expectations. Here's something that you will never see an LLM do but we see humans do all the time: offer an unsolicited response saying that they realized that they made a mistake, or that they have an improvement on their previous argument ... humans do that because they are driven to get things right or to come out ahead in an argument or just to engage in conversation for various reasons... LLMs have no drives, they just match text. An LLM may be able to do great on an LSAT, but it won't pursue a career in law ... it just isn't that sort of thing. Here's what Claude said about it (I think it got it right): The Intentional Stance Overstates LLM Agency Dennett's intentional stance involves treating systems as rational agents with beliefs, desires, and intentions when this proves useful for prediction and explanation. While this might seem to fit LLMs—we naturally say things like "GPT-4 believes X" or "Claude wants to be helpful"—this framing is misleading for several reasons. LLMs lack the continuity of goals and persistent world-models that characterize genuine intentional systems. When an LLM generates a response, it's not pursuing long-term objectives or maintaining beliefs across conversations. Each response emerges from pattern matching against training data, not from reasoning about how to achieve desired outcomes. The "beliefs" we might attribute to an LLM are actually statistical regularities in text, not representations the system uses to guide action. The Design Stance Captures What's Actually Happening The design stance, by contrast, explains behavior by reference to what a system was designed to do and how it was built to function. This fits LLMs perfectly: they were designed to predict and generate text that resembles human language patterns. Their behavior emerges from their training objective (next-token prediction), their architecture (transformer networks), and their training data. When we adopt the design stance toward LLMs, we correctly understand that their seemingly intelligent responses result from sophisticated pattern recognition and statistical inference over vast text corpora. They're not reasoning about beliefs or pursuing goals—they're executing the functions they were designed to perform. This explains both their capabilities (remarkable fluency in text generation) and their limitations (inconsistency, hallucination, lack of genuine understanding). Why This Matters The design stance helps us maintain appropriate expectations about what LLMs can and cannot do, while the intentional stance risks anthropomorphizing systems that, however sophisticated, remain fundamentally different from minds with genuine intentionality.

lo_zamoyski 1y ago

These anthropomorphizations are best described as metaphors when used by people to describe LLMs in common or loose speech. We already use anthropomorphic metaphors when talking about computers. LLMs, like all computation, are a matter of simulation; LLMs can appear to be conversing without actually conversing. What distinguishes the real thing from the simulation is the cause of the appearance of an effect. Problems occur when people forget these words are being used metaphorically, as if they were univocal. Of course, LLMs are multimodal and used to simulate all sorts of things, not just conversation. So there are many possible metaphors we can use, and these metaphors don't necessarily align with the abstractions you might use to talk about LLMs accurately. This is like the difference between "synthesizes text" (abstraction) and "speaks" (metaphor), or "synthesizes images" (abstraction) and "paints" (metaphor). You can use "speaks" or "paints" to talk about the abstractions, of course.

overfeed 1y ago

> We need a higher abstraction level to talk about higher level phenomena in LLMs as well, and the problem is that we have no idea what happens internally at those higher abstraction levels We do know what happens at higher abstraction levels; the design of efficient networks, and the steady beat of SOTA improvements all depend on understanding how LLMs work internally: choice of network dimensions, feature extraction, attention, attention heads, caching, the peculiarities of high-dimensions and avoiding overfitting are all well-understood by practitioners. Anthropomorphization is only necessary in pop-science articles that use a limited vocabulary. IMO, there is very little mystery, but lots of deliberate mysticism, especially about future LLMs - the usual hype-cycle extrapolation.

lawlessone 1y ago

One thing i find i keep forgetting is that asking an LLM why it makes a particular decision is almost pointless. It's reply isn't actually going to be why i did a thing. It's reply is going to be whatever is the most probably string of words that fit as a reason.

amdivia 1y ago

I beg to differ. Anthropomorphizing might blind us to solutions to existing problems. Perhaps instead of trying to come up with the correct prompt for a LLM, there exists a string of words (not necessary ones that make sense) that will get the LLM to a better position to answer given questions. When we anthropomorphize we are inherently ignore certain parts of how LLMs work, and imagining parts that don't even exist

meroes 1y ago
> there exists a string of words (not necessary ones that make sense) that will get the LLM to a better position to answer exactly. The opposite is also true. You might supply more clarifying information to the LLM, which would help any human answer, but it actually degrades the LLM's output.
- mvieira38 1y ago
  This is frequently the case IME, especially with chat interfaces. One or two bad messages and you derail the quality
  - lawlessone 1y ago
    You can just throw in words to bias it towards certain outcomes too. Same applies with image generators or course.

aaroninsf 1y ago

That higher level does exist, indeed a lot philosophy of mind then cognitive science has been investigating exactly this space and devising contested professional nomenclature and modeling about such things for decades now. A useful anchor concept is that of world model, which is what "learning Othello" and similar work seeks to tease out. As someone who worked in precisely these areas for years and has never stopped thinking about them, I find it at turns perplexing, sigh-inducing, and enraging, that the "token prediction" trope gained currency and moreover that it continues to influence people's reasoning about contemporary LLM, often as subtext: an unarticulated fundamental model, which is fundamentally wrong in its critical aspects. It's not that this description of LLM is technically incorrect; it's that it is profoundly _misleading_ and I'm old enough and cynical enough to know full well that many of those who have amplified it and continue to do so, know this very well indeed. Just as the lay person fundamentally misunderstands the relationship between "programming" and these models, and uses slack language in argumentation, the problem with this trope and the reasoning it entails is that what is unique and interesting and valuable about LLM for many applications and interests is how they do what they do. At that level of analysis there is a very real argument to be made that the animal brain is also nothing more than an "engine of prediction," whether the "token" is a byte stream or neural encoding is quite important but not nearly important as the mechanics of the system which operates on those tokens. To be direct, it is quite obvious that LLM have not only vestigial world models, but also self-models; and a general paradigm shift will come around this when multimodal models are the norm: because those systems will share with we animals what philosophers call phenomenology, a model of things as they are "perceived" through the senses. And like we humans, these perceptual models (terminology varies by philosopher and school...) will be bound to the linguistic tokens (both heard and spoken, and written) we attach to them. Vestigial is a key word but an important one. It's not that contemporary LLM have human-tier minds, nor that they have animal-tier world modeling: but they can only "do what they do" because they have such a thing. Of looming importance—something all of us here should set aside time to think about—is that for most reasonable contemporary theories of mind, a self-model embedded in a world-model, with phenomenology and agency, is the recipe for "self" and self-awareness. One of the uncomfortable realities of contemporary LLM already having some vestigial self-model, is that while they are obviously not sentient, nor self-aware, as we are, or even animals are, it is just as obvious (to me at least) that they are self-aware in some emerging sense and will only continue to become more so. Among the lines of finding/research most provocative in this area is the ongoing often sensationalized accounting in system cards and other reporting around two specific things about contemporary models: - they demonstrate behavior pursuing self-preservation - they demonstrate awareness of when they are being tested We don't—collectively or individually—yet know what these things entail, but taken with the assertion that these models are developing emergent self-awareness (I would say: necessarily and inevitably), we are facing some very serious ethical questions. The language adopted by those capitalizing and capitalizing _from_ these systems so far is IMO of deep concern, as it betrays not just disinterest in our civilization collectively benefiting from this technology, but also, that the disregard for human wellbeing implicit in e.g. the hostility to UBI, or, Altman somehow not seeing a moral imperative to remain distant from the current adminstation, implies directly a much greater disregard for "AI wellbeing." That that concept is today still speculative is little comfort. Those of us watching this space know well how fast things are going, and don't mistake plateaus for the end of the curve. I do recommend taking a step back from the line-level grind to give these things some thought. They are going to shape the world we live out our days in and our descendents will spend all of theirs in.

jll29 1y ago

The details in how I talk about LLMs matter. If I use human-related terminology as a shortcut, as some kind of macro to talk at a higher level/more efficiently about something I want to do that might be okay. What is not okay is talking in a way that implies intent, for example. Compare: "The AI doesn't want to do that." versus "The model doesn't do that with this prompt and all others we tried." The latter way of talking is still high-level enough but avoids equating/confusing the name of a field with a sentient being. Whenever I hear people saying "an AI" I suggest they replace AI with "statistics" to make it obvious how problematic anthropomorphisms may have become: *"The statistics doesn't want to do that."

dmitsuki 1y ago
The only reason that sounds weird to you is because you have the experience of being human. Human behavior is not magic. It's still just statistics. You go to the bathroom when you have to pee not because some magical concept of consciousness, but because a reciptor in your brain goes off and starts the chain of making you go to the bathroom. AI's are not magic, but nobody has sufficiently provided any proof we are somehow special either.

TeMPOraL 1y ago

Agreed. I'm also in favor of anthropomorphizing, because not doing so confuses people about the nature and capabilities of these models even more. Whether it's hallucinations, prompt injections, various other security vulnerabilities/scenarios, or problems with doing math, backtracking, getting confused - there's a steady supply of "problems" that some people are surprised to discover and even more surprised this isn't being definitively fixed. Thing is, none of that is surprising, and these things are not bugs, they're flip side of the features - but to see that, one has to realize that humans demonstrate those exact same failure modes. Especially when it comes to designing larger systems incorporating LLM "agents", it really helps to think of them as humans - because the problems those systems face are exactly the same as you get with systems incorporating people, and mostly for the same underlying reasons. Anthropomorphizing LLMs cuts through a lot of misconceptions and false paths, and helps one realize that we have millennia of experience with people-centric computing systems (aka. bureaucracy) that's directly transferrable.

godelski 1y ago
I disagree. Anthropomorphization can be a very useful tool but I think it is currently over used and is a very tricky tool to use when communicating with a more general audience. I think looking at physics might be a good example. We love our simplified examples and there's a big culture of trying to explain things to the lay person (mostly because the topics are incredibly complex). But how many people have misunderstood an observer of a quantum event with "a human" and do not consider "a photon" as an observer? How many people think in Schrodinger's Cat that the cat is both alive and dead?[0] Or believe in a multiverse. There's plenty of examples we can point to. While these analogies *can* be extremely helpful, they *can* also be extremely harmful. This is especially true as information is usually passed through a game of telephone[1]. There is information loss and with it, interpretation becomes more difficult. Often a very subtle part can make a critical distinction. I'm not against anthropomorphization[2], but I do think we should be cautious about how we use it. The imprecise nature of it is the exact reason we should be mindful of when and how to use it. We know that the anthropomorphized analogy is wrong. So we have to think about "how wrong" it is for a given setting. We should also be careful to think about how it may be misinterpreted. That's all I'm trying to say. And isn't this what we should be doing if we want to communicate effectively? [0] It is not. It is either. The point of this thought experiment is that we cannot know the answer without looking inside. There is information loss and the event is not deterministic. It directly relates to the Heisenberg Uncertainty Principle, Godel's Incompleteness, or the Halting Problem. All these things are (loosely) related around the inability to have absolute determinism. [1] https://en.wikipedia.org/wiki/Telephone_game https://en.wikipedia.org/wiki/Telephone_game [2] https://news.ycombinator.com/item?id=44494022 https://news.ycombinator.com/item?id=44494022

seanhunter 1y ago

Exactly. We use anthropomorphic language absolutely all the time when describing different processes for this exact reason - it is a helpful abstraction that allows us to easily describe what’s going on at a high level. “My headphones think they’re connected, but the computer can’t see them”. “The printer thinks it’s out of paper, but it’s not”. “The optimisation function is trying to go down nabla f”. “The parking sensor on the car keeps going off because it’s afraid it’s too close to the wall”. “The client is blocked, because it still needs to get a final message from the server”. …and one final one which I promise you is real because I overheard it “I’m trying to airdrop a photo, but our phones won’t have sex”.

lowsong 1y ago

> The language of "generator that stochastically produces the next word" is just not very useful when you're talking about, e.g., an LLM that is answering complex world modeling questions or generating a creative story. But it isn't modelling. It's been shown time, and time, and time again that LLMs have no internal "model" or "view". This is exactly and precisely why you should not anthropomorphize. And again, the output of an LLM is, by definition, not "creative". Your saying we should anthropomorphize these models when the examples you give are already doing that.