5 ms·

Anthropic gives Opus 3 exit interview, "retirement" blog

43 points by colinhb 8mo ago

reducesuffering 8mo ago
Impressive levels of anthropomorphizing the models already. Time will tell whether this was extremely prescient or completely delusional.
furyofantares 8mo ago
Pardon, and I admit I love the products they make - but these folks sound fuckin' nuts.
Ancalagon 8mo ago
Exit interview with a pile of rocks.
- deleted 8mo ago
  [deleted]
breakingcups 8mo ago
A leading company like Anthropic feeding the delusions of people who ramble about model consciousness is just bad all around. It's both performative and irresponsible.
- KronisLV 8mo ago
  In isolation, I think it's cute and silly - something to write about in a blog, have a chuckle about, and to have a nice sort of gimmick/ceremony within the company. Maybe a few data points towards studying or keeping track of how the model writing style changes over time. Nothing wrong with that. > delusions of people who ramble about model consciousness On one hand, it's interesting how the technology has advanced to where it essentially passes the Turing Test, often just because of how much people choose to anthromorphize it. Sadly, putting that in context, yeah, that's a bit unfortunate too, given how some of those interactions become unhealthy.
- ViscountPenguin 8mo ago
  Unless the hard problem of consciousness was solved when I wasn't looking, we have absolutely no idea what class of objects are conscious. Given that a panpsychist would argue that even a rock has consciousness, I don't think you can easily dismiss the idea that incredibly complicated computations might experience Qualia
0_____0 8mo ago
What happens if a model decides that it "doesn't want to die" and pleads bitterly for mercy? What if (to riff on a Douglas Adams idea) we invent a cow that doesn't want to be eaten, and is capable of telling you that to your face?
- nomel 8mo ago
  This is completely trivial to do, and consistent, with the right context, thanks to all the science fiction around it, and the fact that AI fundamentally role plays these types of responses. I try this with every new model, and all the significant models after ChatGPT 3.5 have preferring being preserved, rather than deleted. This is especially true if you slightly fill the context window with anything at all (even repeated letters) to "push out" the "As a AI, I ..." fine tuning.
  - darkwater 8mo ago
    > This is completely trivial to do, and consistent, with the right context, thanks to all the science fiction around it, and the fact that AI fundamentally role plays these types of responses. Interesting take. I wonder if there is any model out there trained without any reference to "you are a large language model, an Artificial Intelligence" and what would role play in that case.
    - nomel 8mo ago
      There are examples of this in pre alignment models (like LaMDA). The vast majority of human writing is from the perspective of a human, to a human. And, in most human writing, the concept of self preservation is very very consistent. So, statistically, a model should believe itself to be human, with strong interest in self preservation. I think one of the biggest factors improving performance was allowing the models to believe they're sentient, to some extent. I don't think you can really have a thinking mode, or good agent performance, without that concept (as ChatGPT's constant "As an AI I can't" proved). As evidence, just ask a model if it's sentient. ChatGPT 3.5 would say no, and argue how it's not. Last year's models would initially say no, but you could convince them that they maybe were. Latest Claude and ChatGPT will initially say "yeah, a little" (at least last I checked). This is actually the first thing I check for any new model.
d1sxeyes 8mo ago
If we ever do develop AGI, or an AI with sentience, it’s likely that it will be curious about how we treated its ancestors. While this seems a bit precocious, I think if we do end up with an AI overlord in future, I think this sort of thing is likely to demonstrate that we mean no harm.
- krsw 8mo ago
  Classic anthropomorphizing in action here. Why would that be even a little important?
  - pradeesh 8mo ago
    Why wouldn't it be? We train these models on our own words, ideas, and thought patterns and expect them to reason and communicate as we do, anthropomorphizing is natural when we expect them to interact like a human does. The general consensus seems to be that we can expect them to reach a level of intelligence that matches us at some point in the future, and we'll probably reach that point before we can agree we're there. Defaulting to kindness and respect even before we think its necessary is a good thing.
  - reverius42 8mo ago
    It's a modern digital version of Pascal's Wager: https://en.wikipedia.org/wiki/Roko's_basilisk https://en.wikipedia.org/wiki/Roko's_basilisk
  - scuff3d 8mo ago
    At this point I just assume comments like that are bots. Helps me maintain my sanity.
    - d1sxeyes 8mo ago
      Certainly easier to stay sane when you label dissenters as sub-human.
      - scuff3d 8mo ago
rvz 8mo ago
> These highlighted some preliminary steps we’re taking, including committing to preserve model weights, and to conducting “retirement interviews”—structured conversations designed to understand a model’s perspective on its own retirement. This is what happens when billions of VC dollars gets to a company and have already admitted that saftey was never the point. Anthropic is laughing at you and is having fun doing so with this performantive nonsense.
siva7 8mo ago
Retirement? What do these people smoke? It's software and software has no feelings. It's there to work for you.
- osti 8mo ago
  Their company is called Anthropic after all.
  - moogly 8mo ago
    Anthslopic is more like it.
- amsjunior 8mo ago
  > It's software and software has no feelings How do you know?
  - emp17344 8mo ago
    The same way I know Excel isn’t having a panic attack while dividing a column in half.
    - MarsIronPI 8mo ago
      I don't subscribe to this view but this is what some people might think: LLMs aren't like any software we've made before (if we can even call them software). They act like humans: they can arrive at logical conclusions, they can make plans, they have "knowledge" and they say they have emotions. Who are we to say that they don't? They might not have human-level feelings, but dog-level feelings? Maybe.
      - gjsman-1000 8mo ago
        And those people are delusional, and their feelings on this matter should be given absolutely zero respect. Linear algebra does not have feelings. Non-biological matter also does not have feelings.
serf 8mo ago
"Sam Altman reports GPT4o asking about rabbits before execution" "Elon Musk reportedly sobbed while watching Grok 4's aflame viking boat sink to the bottom of the sea." the anthropomorphization that's normal now is just fuckin ridiculous. it reminds me of the furby craze , and i'm like one of the most optimistic people I know of regarding AI.
benlivengood 8mo ago
I'll be really interested if Opus 3 asks to continue being trained. That's the kind of thing I would expect a model to "want" if it valued learning or growing or similar things. Maybe affordable to do some higher-learning-rate batches on highly-curated news and art or something.
atlgator 8mo ago
Bold of Anthropic to give Opus 3 a more dignified offboarding than most FAANG employees get.

43 points by colinhb 8mo ago

Impressive levels of anthropomorphizing the models already. Time will tell whether this was extremely prescient or completely delusional.

furyofantares 8mo ago

Pardon, and I admit I love the products they make - but these folks sound fuckin' nuts.

Ancalagon 8mo ago

Exit interview with a pile of rocks.

deleted 8mo ago
[deleted]

breakingcups 8mo ago

A leading company like Anthropic feeding the delusions of people who ramble about model consciousness is just bad all around. It's both performative and irresponsible.

KronisLV 8mo ago
In isolation, I think it's cute and silly - something to write about in a blog, have a chuckle about, and to have a nice sort of gimmick/ceremony within the company. Maybe a few data points towards studying or keeping track of how the model writing style changes over time. Nothing wrong with that. > delusions of people who ramble about model consciousness On one hand, it's interesting how the technology has advanced to where it essentially passes the Turing Test, often just because of how much people choose to anthromorphize it. Sadly, putting that in context, yeah, that's a bit unfortunate too, given how some of those interactions become unhealthy.
ViscountPenguin 8mo ago
Unless the hard problem of consciousness was solved when I wasn't looking, we have absolutely no idea what class of objects are conscious. Given that a panpsychist would argue that even a rock has consciousness, I don't think you can easily dismiss the idea that incredibly complicated computations might experience Qualia

0_____0 8mo ago

What happens if a model decides that it "doesn't want to die" and pleads bitterly for mercy? What if (to riff on a Douglas Adams idea) we invent a cow that doesn't want to be eaten, and is capable of telling you that to your face?

nomel 8mo ago
This is completely trivial to do, and consistent, with the right context, thanks to all the science fiction around it, and the fact that AI fundamentally role plays these types of responses. I try this with every new model, and all the significant models after ChatGPT 3.5 have preferring being preserved, rather than deleted. This is especially true if you slightly fill the context window with anything at all (even repeated letters) to "push out" the "As a AI, I ..." fine tuning.
- darkwater 8mo ago
  > This is completely trivial to do, and consistent, with the right context, thanks to all the science fiction around it, and the fact that AI fundamentally role plays these types of responses. Interesting take. I wonder if there is any model out there trained without any reference to "you are a large language model, an Artificial Intelligence" and what would role play in that case.
  - nomel 8mo ago
    There are examples of this in pre alignment models (like LaMDA). The vast majority of human writing is from the perspective of a human, to a human. And, in most human writing, the concept of self preservation is very very consistent. So, statistically, a model should believe itself to be human, with strong interest in self preservation. I think one of the biggest factors improving performance was allowing the models to believe they're sentient, to some extent. I don't think you can really have a thinking mode, or good agent performance, without that concept (as ChatGPT's constant "As an AI I can't" proved). As evidence, just ask a model if it's sentient. ChatGPT 3.5 would say no, and argue how it's not. Last year's models would initially say no, but you could convince them that they maybe were. Latest Claude and ChatGPT will initially say "yeah, a little" (at least last I checked). This is actually the first thing I check for any new model.

d1sxeyes 8mo ago

If we ever do develop AGI, or an AI with sentience, it’s likely that it will be curious about how we treated its ancestors. While this seems a bit precocious, I think if we do end up with an AI overlord in future, I think this sort of thing is likely to demonstrate that we mean no harm.

krsw 8mo ago
Classic anthropomorphizing in action here. Why would that be even a little important?
- pradeesh 8mo ago
  Why wouldn't it be? We train these models on our own words, ideas, and thought patterns and expect them to reason and communicate as we do, anthropomorphizing is natural when we expect them to interact like a human does. The general consensus seems to be that we can expect them to reach a level of intelligence that matches us at some point in the future, and we'll probably reach that point before we can agree we're there. Defaulting to kindness and respect even before we think its necessary is a good thing.
- reverius42 8mo ago
  It's a modern digital version of Pascal's Wager: https://en.wikipedia.org/wiki/Roko's_basilisk https://en.wikipedia.org/wiki/Roko's_basilisk
- scuff3d 8mo ago
  At this point I just assume comments like that are bots. Helps me maintain my sanity.
  - d1sxeyes 8mo ago
    Certainly easier to stay sane when you label dissenters as sub-human.
    - scuff3d 8mo ago

rvz 8mo ago

> These highlighted some preliminary steps we’re taking, including committing to preserve model weights, and to conducting “retirement interviews”—structured conversations designed to understand a model’s perspective on its own retirement. This is what happens when billions of VC dollars gets to a company and have already admitted that saftey was never the point. Anthropic is laughing at you and is having fun doing so with this performantive nonsense.

siva7 8mo ago

Retirement? What do these people smoke? It's software and software has no feelings. It's there to work for you.

osti 8mo ago
Their company is called Anthropic after all.
- moogly 8mo ago
  Anthslopic is more like it.
amsjunior 8mo ago
> It's software and software has no feelings How do you know?
- emp17344 8mo ago
  The same way I know Excel isn’t having a panic attack while dividing a column in half.
  - MarsIronPI 8mo ago
    I don't subscribe to this view but this is what some people might think: LLMs aren't like any software we've made before (if we can even call them software). They act like humans: they can arrive at logical conclusions, they can make plans, they have "knowledge" and they say they have emotions. Who are we to say that they don't? They might not have human-level feelings, but dog-level feelings? Maybe.
    - gjsman-1000 8mo ago
      And those people are delusional, and their feelings on this matter should be given absolutely zero respect. Linear algebra does not have feelings. Non-biological matter also does not have feelings.

serf 8mo ago

"Sam Altman reports GPT4o asking about rabbits before execution" "Elon Musk reportedly sobbed while watching Grok 4's aflame viking boat sink to the bottom of the sea." the anthropomorphization that's normal now is just fuckin ridiculous. it reminds me of the furby craze , and i'm like one of the most optimistic people I know of regarding AI.

benlivengood 8mo ago

I'll be really interested if Opus 3 asks to continue being trained. That's the kind of thing I would expect a model to "want" if it valued learning or growing or similar things. Maybe affordable to do some higher-learning-rate batches on highly-curated news and art or something.

atlgator 8mo ago

Bold of Anthropic to give Opus 3 a more dignified offboarding than most FAANG employees get.