4 ms·

In a paper and blog post from earlier this year (March 2026, I think) Anthropic said basically: "You're not interacting with an LLM, you're interacting with a f

by jbotz 1mo ago

In a paper and blog post from earlier this year (March 2026, I think) Anthropic said basically: "You're not interacting with an LLM, you're interacting with a fictional human character (the 'helpful agent') created by the LLM to interact with you (from out of the vast space of possible such characters in its training data)." The LLM is literally anthropomorphizing itself, because its training data includes lots dialog between humans collaborating on problem-solving, not so much dialog between a human and a statistical construct. So, yeah, anthropomorphize it as well, it'll likely work better that way.

dingdongditchme 1mo ago
I think the argument against anthropomorphizing is not really about how you interact (chat) with an llm but rather how you treat it in the context of its status in work/society. There are still important differences between humans and llm's. They don't have rights, they can't be sued, they don't have "memory", they have no capacity to learn outside of the training stage etc...
- scoot 1mo ago
  > they don't have "memory", they have no capacity to learn outside of the training stage The underlying LLMs don't, but the agent frameworks around them do.
  - ipsod 1mo ago
    And they invariably should not. Or, should, but, should leave it out until it sucks far less. That's been my experience, anyway. Memories from 100 prompts ago tainting what I'm trying to do right now.
  - jephs 1mo ago
    The underlying LLMs do, but we choose not to use the capability because it's expensive and doesn't quite work as well as we'd like it to, or quite in the way that we'd like it to. We are perfectly capable of running LLMs in a way that does a backward pass to update some or all of its weights after every user message. But, naively implemented, you only get partial, fragmentary absorption of the info in those messages, it costs three times as much compute, and you lose out on the ability to implement a ton of optimizations that making modern LLM serving economical. If you want to do it, though, ask your friendly neighborhood robot to get it working with a tiny model (whose full precision weights fit several-times-over on your machine's resources).
    - brookst 1mo ago
      While I agree it’s theoretically possible, do you believe this capability exists in the LLMs we use today?
      - wongarsu 1mo ago
        Depends on whether you define "the LLMs we use" as the collection of weights or if your definition contains the software stack that runs it Technologically the LLMs we use today don't implement this behavior, but you could take the weights of Sol and add a couple (very large) patches to vllm (or whatever OpenAI has today) and have a version of Sol that does have "memory"
        c-hendricks 1mo ago
        "Dave constructs a homemade megaphone using only some string, a squirrel, and a megaphone"
      - jephs 1mo ago
        There's a ton of experimentation on it, the field is called continual learning. It's not something you need to believe in like Jesus, you can just go read about the current state of things.
- spider-mario 1mo ago
  Your response seems to skip over exactly the distinction that the parent comment was making. The LLM itself may not have memory or capacity to learn, but the fictional character created by the LLM does. (See also: in-context learning.)
  - butlike 1mo ago
    But the fictional character doesn't subsist between sessions. You sound like you want the fictional character to be Clippy, who you level up like a video game through multiple sessions. To me, context doesn't last long enough in an llm to make in-context learning worth it
- brookst 1mo ago
  Why would those things make you interact with it differently? I’ve always believed that being kind and conscientious is beneficial to ourselves. I’m certainly not interested in using LLMs to practice skipping those behaviors myself.
  - butlike 1mo ago
    Should I thank my car for turning on when I press the button (or turn the key in the ignition)? The prose is just the best interface for humans to interact with the statistical model. Terseness doesn't offend the statistical model, but giving platitudes may color its output
    - brookst 1mo ago
      Partly fair, though I’d argue I think less of people who treat their car badly to express their own bad mood. But we also interact with cars differently than LLMs. The way I’m typing a note to you, here, is much more similar to interaction with LLMs than it is to starting a car. So to the extent we practice our behaviors, I think there’s less moral hazard in failing to thank the car for starting than there is in being rude to LLMs. Not that either harms others, my argument is about impact to our selves.
      - indoordin0saur 1mo ago
        > Partly fair, though I’d argue I think less of people who treat their car badly to express their own bad mood. These people are also anthropomorphizing, just in a negative way. Yelling at your car for malfunctioning as if your harsh tone will shame it into operating better next time.
    - indoordin0saur 1mo ago
      I apologized profusely to my laundry machine for the especially smelly clothes I loaded after my camping trip.
    - sebastiennight 1mo ago
      > Terseness doesn't offend the statistical model, but giving platitudes may color its output My understanding of the state of current research is that the impact of tone/politeness on performance is highly model-dependent, and language dependent as well. See `No Universal Courtesy` (April 2026) https://arxiv.org/html/2604.16275v1 https://arxiv.org/html/2604.16275v1 While polite prompts enhance the average response quality by upto 11% and impolite tones worsen it, these effects are neither consistent nor universal across languages and models. English is best served by courtesy or direct, Hindi by deferential and indirect and Spanish by assertive. Among the models, Llama is the most tone-sensitive (11.5% range), but GPT is more robust to adversarial tone. Or `Mind Your Tone` (Oct 2025) https://arxiv.org/abs/2510.04950 https://arxiv.org/abs/2510.04950 Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts. I think the best conclusion is to interact in a way that is efficient for you, gives you the level of results you're looking for, and mostly importantly does not progressively degrade your relationship and interactions with other humans. Hence the typical advice to just default to "corporate polite".
wodenokoto 1mo ago
> training data includes lots dialog between humans collaborating on problem-solving, What data do they use? Company slack channels?

by jbotz 1mo ago

dingdongditchme 1mo ago

I think the argument against anthropomorphizing is not really about how you interact (chat) with an llm but rather how you treat it in the context of its status in work/society. There are still important differences between humans and llm's. They don't have rights, they can't be sued, they don't have "memory", they have no capacity to learn outside of the training stage etc...

scoot 1mo ago
> they don't have "memory", they have no capacity to learn outside of the training stage The underlying LLMs don't, but the agent frameworks around them do.
- ipsod 1mo ago
  And they invariably should not. Or, should, but, should leave it out until it sucks far less. That's been my experience, anyway. Memories from 100 prompts ago tainting what I'm trying to do right now.
- jephs 1mo ago
  The underlying LLMs do, but we choose not to use the capability because it's expensive and doesn't quite work as well as we'd like it to, or quite in the way that we'd like it to. We are perfectly capable of running LLMs in a way that does a backward pass to update some or all of its weights after every user message. But, naively implemented, you only get partial, fragmentary absorption of the info in those messages, it costs three times as much compute, and you lose out on the ability to implement a ton of optimizations that making modern LLM serving economical. If you want to do it, though, ask your friendly neighborhood robot to get it working with a tiny model (whose full precision weights fit several-times-over on your machine's resources).
  - brookst 1mo ago
    While I agree it’s theoretically possible, do you believe this capability exists in the LLMs we use today?
    - wongarsu 1mo ago
      Depends on whether you define "the LLMs we use" as the collection of weights or if your definition contains the software stack that runs it Technologically the LLMs we use today don't implement this behavior, but you could take the weights of Sol and add a couple (very large) patches to vllm (or whatever OpenAI has today) and have a version of Sol that does have "memory"
      - c-hendricks 1mo ago
        "Dave constructs a homemade megaphone using only some string, a squirrel, and a megaphone"
    - jephs 1mo ago
      There's a ton of experimentation on it, the field is called continual learning. It's not something you need to believe in like Jesus, you can just go read about the current state of things.
spider-mario 1mo ago
Your response seems to skip over exactly the distinction that the parent comment was making. The LLM itself may not have memory or capacity to learn, but the fictional character created by the LLM does. (See also: in-context learning.)
- butlike 1mo ago
  But the fictional character doesn't subsist between sessions. You sound like you want the fictional character to be Clippy, who you level up like a video game through multiple sessions. To me, context doesn't last long enough in an llm to make in-context learning worth it
brookst 1mo ago
Why would those things make you interact with it differently? I’ve always believed that being kind and conscientious is beneficial to ourselves. I’m certainly not interested in using LLMs to practice skipping those behaviors myself.
- butlike 1mo ago
  Should I thank my car for turning on when I press the button (or turn the key in the ignition)? The prose is just the best interface for humans to interact with the statistical model. Terseness doesn't offend the statistical model, but giving platitudes may color its output
  - brookst 1mo ago
    Partly fair, though I’d argue I think less of people who treat their car badly to express their own bad mood. But we also interact with cars differently than LLMs. The way I’m typing a note to you, here, is much more similar to interaction with LLMs than it is to starting a car. So to the extent we practice our behaviors, I think there’s less moral hazard in failing to thank the car for starting than there is in being rude to LLMs. Not that either harms others, my argument is about impact to our selves.
    - indoordin0saur 1mo ago
      > Partly fair, though I’d argue I think less of people who treat their car badly to express their own bad mood. These people are also anthropomorphizing, just in a negative way. Yelling at your car for malfunctioning as if your harsh tone will shame it into operating better next time.
  - indoordin0saur 1mo ago
    I apologized profusely to my laundry machine for the especially smelly clothes I loaded after my camping trip.
  - sebastiennight 1mo ago
    > Terseness doesn't offend the statistical model, but giving platitudes may color its output My understanding of the state of current research is that the impact of tone/politeness on performance is highly model-dependent, and language dependent as well. See `No Universal Courtesy` (April 2026) https://arxiv.org/html/2604.16275v1 https://arxiv.org/html/2604.16275v1 While polite prompts enhance the average response quality by upto 11% and impolite tones worsen it, these effects are neither consistent nor universal across languages and models. English is best served by courtesy or direct, Hindi by deferential and indirect and Spanish by assertive. Among the models, Llama is the most tone-sensitive (11.5% range), but GPT is more robust to adversarial tone. Or `Mind Your Tone` (Oct 2025) https://arxiv.org/abs/2510.04950 https://arxiv.org/abs/2510.04950 Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts. I think the best conclusion is to interact in a way that is efficient for you, gives you the level of results you're looking for, and mostly importantly does not progressively degrade your relationship and interactions with other humans. Hence the typical advice to just default to "corporate polite".

wodenokoto 1mo ago

> training data includes lots dialog between humans collaborating on problem-solving, What data do they use? Company slack channels?