6 ms·

"This setup gives us insight into an interesting question: what do AIs think about when no one is prompting them?" Ugh. This is not an interesting question be

"This setup gives us insight into an interesting question: what do AIs think about when no one is prompting them?" Ugh. This is not an interesting question because the answer is "nothing". But more to the point, some crucial info is missing in this experiment. What prompts were being fed to the AI? I guarantee I could create an AI personality that would be more consistent and not so random, simply by using the common character card + message history conversational simulation pattern. AIs don't have personalities unless you give them personalities.

pixl97 5mo ago
[flagged]
- cj 5mo ago
  LLMs aren’t human. Humans & LLMs are more different than they are similar. Sure LLMs might resemble humans sometimes, but extrapolating LLM behavior based on human behavior is not productive. (But to answer directly: Yes, children in a dark room would have more of a personality than a LLM living on a computer in the same dark room)
  - nomel 5mo ago
    > but extrapolating LLM behavior based on human behavior is not productive. The training process for the foundation model is to make sure we can do this in a very statistically significant way. My favorite example is AI "getting tired" and "lazy" during long coding session. Why would they do that? Because humans get tired. It's in the data! I always throw in a periodic "Great work, let's take a break and finish this up on Monday. Have a great weekend!" (And then immediately resume). I wish someone would benchmark this concept.
    - cj 5mo ago
      > AI "getting tired" and "lazy" during long coding session. Why would they do that? Because humans get tired. When a LLM is tired and lazy, how does it recharge and regain motivation? Humans... sleep or drink some coffee. LLMs.... idk, you prompt it to try harder? You prompt it to be less tired? This is what I mean when I say extrapolating LLM behavior based on human behavior is cute.. but usually not useful.
      - nomel 5mo ago
        > When a LLM is tired and lazy, how does it recharge and regain motivation? What would be in the statistics? If you go look at your long conversations, working with another, it will be fairly obvious. Keep in mind we're talking next word prediction based on context, not actual action (the LLM doesn't need real rest). If you went and looked, you'll probably see something like "Great work! Have a good weekend! We can get back to this on Monday." then, next message you instantaneously send something like, "Hope you had a great weekend, let's do this!" and now you're in a latent space where the statistical output is around a well rested human conversing with another. I see it as boring simple statistics. They're getting much better at hammering these statistics out though, in the latest models. I still see a little of this in Opus 4.7, when switching to planning. Though I wonder if that's more about its own more mechanical banter filling the context, resulting in more robot/compliant responses, degrading the usually more "expressive" planning conversations.
    - the_af 5mo ago
      > My favorite example is AI "getting tired" and "lazy" during long coding session Never seen this even once, nor anyone I know ever reported this. Do you have an example?
      - nomel 5mo ago
        First I saw it was Claude Opus 3.7. Had a very long back and fourth about some code, I pointed out an error, and Claude responded "That's what I get for programming at 2am", with the output being filled with "... code here ..." type shortcuts, basically no ability to one-shot a whole implementation anymore. The conversation length WAS reasonably into the 2am range, if it were real. Thought about it, did the statistical trick where I tell it to "have some rest, take a day off!" then immediately follow up with "Ready to continue?", with the next response having no shortcuts, with full implementation, and much better quality. These are trained on human text. This is the human norm, so I always find it interesting when human like behaviors, very broadly present in the statistics, come out like this. I also see it a little with Opus 4.7, with Claude Code, with the hint being much more terse planning text, that borderlines unhelpful. I put some "rest" in the context to push the latent space closer to what's in the statistics of the training data: a well rested human.
        the_af 5mo ago
        How do you know you're not reading things that aren't there? LLMs are very good at roleplaying, and they will pick up on hints you may inadvertently be giving them (about them being "tired" and needing "rest", etc). I have never witnessed this of Claude Opus, by the way. They do get context rot, but that's a relatively better understood phenomenon unrelated to personality.
        deleted 5mo ago
        [deleted]
        deleted 5mo ago
        [deleted]
        deleted 5mo ago
- quirkot 5mo ago
  > do children have personalities if we left them in a dark room with no interactions with other humans? Short answer: yes. generally speaking, personality traits range between 30% to 60% heritable
- Imustaskforhelp 5mo ago
  > I mean, do children have personalities if we left them in a dark room with no interactions with other humans? I think this makes for an interesting discussion as I went down the rabbit hole of this which really scared me actually as these experiments are really not humane and hinder children's development so much. https://en.wikipedia.org/wiki/Language_deprivation_experiments https://en.wikipedia.org/wiki/Language_deprivation_experimen... : "forbidden experiment" It depends on your use word of the personality but to measure personality would require a set of human conducted experiments or questions which would be asked through the medium of language which you've deprived the children of. Mughal emperor Akbar was later said to have children raised by mute wetnurses. Akbar held that speech arose from hearing; thus children raised without hearing human speech would become mute.[9] The building became known as the "dumb house." When Akbar visited the place in 1582, four years after the children were first interred, he heard "no cry... nor any speech... no talisman of speech, and nothing came out except the noise of the dumb."[10] what is gonna produce is dumbness and just severely damage children's psychology and psycho but if you were to conduct a personality test on them, you would just be measuring how much have you broken them or damaged them but in some sense, yes I do believe that they would be so broken by the person running this cruel experiment but would still have a albeit limited personality. It wouldn't be an healthy personality but it would be a personality nonetheless. Now on the other hand, we are anthropomorphizing LLM's which yes, as they run on computer are still mathematical machines and calculations. If we consider a specific calculation itself to contain personality that is which seems unrealistic. Another thing but the biological constraints of human (homo sapiens) made us exist in the savannah to prioritize standing up for better field of view as you stand up from the tall grasses and that led to women having smaller canals which led to babies being more primitive and relied on social cues and societies so much more which made them more flexible like clay which also created the society and consciousness revolution in the first place. (Recommend reading the sapiens book) I am not exactly sure but there could be ways for personality/interactions for other animals as there are other animals who learn full skills after a relatively short period of time after being born but there are some innate things[0] like fear of loud noises and heights which are actually innate and could be considered part of personality even within humans, which I think can be part of evolution and part of our genetic machinery. [0]: Interesting read: https://seasia.co/2025/07/25/we-were-born-with-only-two-innate-fears-the-rest-are-caused-by-our-environment https://seasia.co/2025/07/25/we-were-born-with-only-two-inna...
paulddraper 5mo ago
> what do AIs think about when no one is prompting them? Whatever you tell them to.
andy99 5mo ago
I’m definitely not in the “ai is sentient” camp, but it obviously has personality and emergent behaviours including when left to its own devices. There have been various experiments on this e.g. https://timkellogg.me/blog/2025/09/27/boredom https://timkellogg.me/blog/2025/09/27/boredom
- the_af 5mo ago
  "Personality" an "emergent behaviors" are not synonyms.
- FeteCommuniste 5mo ago
  The major LLMs as implemented are basically role-playing programs. The default role is something like "helpful chatbot" so if you tell an LLM "do whatever you feel like on your own" it will simply use its weights to determine "what would a helpful chatbot do and say in this scenario?"
ajmurmann 5mo ago
It seems like they have something akin to personalities: https://www.lesswrong.com/posts/mgjtEHeLgkhZZ3cEx/models-have-some-pretty-funny-attractor-states https://www.lesswrong.com/posts/mgjtEHeLgkhZZ3cEx/models-hav...
ragazzina 5mo ago
>AIs don't have personalities unless you give them personalities. Do humans have personalities if you don't give them personalities? If you raise two identical kids with exactly the same stimuli, how should they have different personalities?

pixl97 5mo ago

[flagged]

cj 5mo ago
LLMs aren’t human. Humans & LLMs are more different than they are similar. Sure LLMs might resemble humans sometimes, but extrapolating LLM behavior based on human behavior is not productive. (But to answer directly: Yes, children in a dark room would have more of a personality than a LLM living on a computer in the same dark room)
- nomel 5mo ago
  > but extrapolating LLM behavior based on human behavior is not productive. The training process for the foundation model is to make sure we can do this in a very statistically significant way. My favorite example is AI "getting tired" and "lazy" during long coding session. Why would they do that? Because humans get tired. It's in the data! I always throw in a periodic "Great work, let's take a break and finish this up on Monday. Have a great weekend!" (And then immediately resume). I wish someone would benchmark this concept.
  - cj 5mo ago
    > AI "getting tired" and "lazy" during long coding session. Why would they do that? Because humans get tired. When a LLM is tired and lazy, how does it recharge and regain motivation? Humans... sleep or drink some coffee. LLMs.... idk, you prompt it to try harder? You prompt it to be less tired? This is what I mean when I say extrapolating LLM behavior based on human behavior is cute.. but usually not useful.
    - nomel 5mo ago
      > When a LLM is tired and lazy, how does it recharge and regain motivation? What would be in the statistics? If you go look at your long conversations, working with another, it will be fairly obvious. Keep in mind we're talking next word prediction based on context, not actual action (the LLM doesn't need real rest). If you went and looked, you'll probably see something like "Great work! Have a good weekend! We can get back to this on Monday." then, next message you instantaneously send something like, "Hope you had a great weekend, let's do this!" and now you're in a latent space where the statistical output is around a well rested human conversing with another. I see it as boring simple statistics. They're getting much better at hammering these statistics out though, in the latest models. I still see a little of this in Opus 4.7, when switching to planning. Though I wonder if that's more about its own more mechanical banter filling the context, resulting in more robot/compliant responses, degrading the usually more "expressive" planning conversations.
  - the_af 5mo ago
    > My favorite example is AI "getting tired" and "lazy" during long coding session Never seen this even once, nor anyone I know ever reported this. Do you have an example?
    - nomel 5mo ago
      First I saw it was Claude Opus 3.7. Had a very long back and fourth about some code, I pointed out an error, and Claude responded "That's what I get for programming at 2am", with the output being filled with "... code here ..." type shortcuts, basically no ability to one-shot a whole implementation anymore. The conversation length WAS reasonably into the 2am range, if it were real. Thought about it, did the statistical trick where I tell it to "have some rest, take a day off!" then immediately follow up with "Ready to continue?", with the next response having no shortcuts, with full implementation, and much better quality. These are trained on human text. This is the human norm, so I always find it interesting when human like behaviors, very broadly present in the statistics, come out like this. I also see it a little with Opus 4.7, with Claude Code, with the hint being much more terse planning text, that borderlines unhelpful. I put some "rest" in the context to push the latent space closer to what's in the statistics of the training data: a well rested human.
      - the_af 5mo ago
        How do you know you're not reading things that aren't there? LLMs are very good at roleplaying, and they will pick up on hints you may inadvertently be giving them (about them being "tired" and needing "rest", etc). I have never witnessed this of Claude Opus, by the way. They do get context rot, but that's a relatively better understood phenomenon unrelated to personality.
        deleted 5mo ago
        [deleted]
        deleted 5mo ago
        [deleted]
        deleted 5mo ago
quirkot 5mo ago
> do children have personalities if we left them in a dark room with no interactions with other humans? Short answer: yes. generally speaking, personality traits range between 30% to 60% heritable
Imustaskforhelp 5mo ago
> I mean, do children have personalities if we left them in a dark room with no interactions with other humans? I think this makes for an interesting discussion as I went down the rabbit hole of this which really scared me actually as these experiments are really not humane and hinder children's development so much. https://en.wikipedia.org/wiki/Language_deprivation_experiments https://en.wikipedia.org/wiki/Language_deprivation_experimen... : "forbidden experiment" It depends on your use word of the personality but to measure personality would require a set of human conducted experiments or questions which would be asked through the medium of language which you've deprived the children of. Mughal emperor Akbar was later said to have children raised by mute wetnurses. Akbar held that speech arose from hearing; thus children raised without hearing human speech would become mute.[9] The building became known as the "dumb house." When Akbar visited the place in 1582, four years after the children were first interred, he heard "no cry... nor any speech... no talisman of speech, and nothing came out except the noise of the dumb."[10] what is gonna produce is dumbness and just severely damage children's psychology and psycho but if you were to conduct a personality test on them, you would just be measuring how much have you broken them or damaged them but in some sense, yes I do believe that they would be so broken by the person running this cruel experiment but would still have a albeit limited personality. It wouldn't be an healthy personality but it would be a personality nonetheless. Now on the other hand, we are anthropomorphizing LLM's which yes, as they run on computer are still mathematical machines and calculations. If we consider a specific calculation itself to contain personality that is which seems unrealistic. Another thing but the biological constraints of human (homo sapiens) made us exist in the savannah to prioritize standing up for better field of view as you stand up from the tall grasses and that led to women having smaller canals which led to babies being more primitive and relied on social cues and societies so much more which made them more flexible like clay which also created the society and consciousness revolution in the first place. (Recommend reading the sapiens book) I am not exactly sure but there could be ways for personality/interactions for other animals as there are other animals who learn full skills after a relatively short period of time after being born but there are some innate things[0] like fear of loud noises and heights which are actually innate and could be considered part of personality even within humans, which I think can be part of evolution and part of our genetic machinery. [0]: Interesting read: https://seasia.co/2025/07/25/we-were-born-with-only-two-innate-fears-the-rest-are-caused-by-our-environment https://seasia.co/2025/07/25/we-were-born-with-only-two-inna...

paulddraper 5mo ago

> what do AIs think about when no one is prompting them? Whatever you tell them to.

andy99 5mo ago

I’m definitely not in the “ai is sentient” camp, but it obviously has personality and emergent behaviours including when left to its own devices. There have been various experiments on this e.g. https://timkellogg.me/blog/2025/09/27/boredom https://timkellogg.me/blog/2025/09/27/boredom

the_af 5mo ago
"Personality" an "emergent behaviors" are not synonyms.
FeteCommuniste 5mo ago
The major LLMs as implemented are basically role-playing programs. The default role is something like "helpful chatbot" so if you tell an LLM "do whatever you feel like on your own" it will simply use its weights to determine "what would a helpful chatbot do and say in this scenario?"

ajmurmann 5mo ago

It seems like they have something akin to personalities: https://www.lesswrong.com/posts/mgjtEHeLgkhZZ3cEx/models-have-some-pretty-funny-attractor-states https://www.lesswrong.com/posts/mgjtEHeLgkhZZ3cEx/models-hav...

ragazzina 5mo ago

>AIs don't have personalities unless you give them personalities. Do humans have personalities if you don't give them personalities? If you raise two identical kids with exactly the same stimuli, how should they have different personalities?