4 ms·

I object to "roleplaying", because that assumes there's even a cohesive entity that's capable of "pretending" in the first place. There isn't, the LLM algorithm

by Terr_ 3mo ago

I object to "roleplaying", because that assumes there's even a cohesive entity that's capable of "pretending" in the first place. There isn't, the LLM algorithm is a document generator, a (really awesome) mad-libs device. When the incremental output looks like a first-person story the algorithm is not "roleplaying" the character/narrator. Similarly, when it looks like a spreadsheet, it's not "being numbers", and when it's a picture of a flower it's not "becoming one with nature." etc. Insofar as any roleplaying is happening, it's being done by humans as we observe! We read text, perceive a story and fictional characters and instinctively start imagining and simulating minds (mostly like our own) that never really existed, taking cues from descriptions and dialogue. Those characters' "motivations" and "thoughts" are just as imagined as their noses or clothing or birth-dates. This is Plato's cave [0] stuff, where we see a shadow like a person, assume a person is there, but ultimately it's paper shape on a stick. We can't help "the person" solve their eating-disorder, because they never existed, we can only trim the paper cutout. [0] https://en.wikipedia.org/wiki/Allegory_of_the_cave https://en.wikipedia.org/wiki/Allegory_of_the_cave

cyanydeez 3mo ago
theyre litterally trained on " you are an assistant"
- Terr_ 3mo ago
  Literally correct, but you're looking at the wrong layer of abstraction. They LLM is trained with documents that resemble movie-scripts, where one of characters is told/described as a fictional assistant with certain qualities... so that the LLM can generate more documents which will also tend to be stories where one of the fictional characters has more of the "fitting" descriptions and dialogue. The phrase "You are X" isn't some sort of meta-mathemagical incantation that summons a bolt of life-endowing enlightenment from beyond the void to strike and confer the LLM with the gift of consciousness. [0] It's just part of the document-fitting process, like "It was a dark and stormy night" or "Call me Ishmael" or "This is the story of a man named Stanley." [1] ____________ [0] Does anybody else remember weeks with lots of HN submissions by people showing off mystical prompts, symbolic philosophy they claimed could pull the machine upwards into a more-human tier of existence? [1] https://www.youtube.com/watch?v=fBtX0S2J32Y https://www.youtube.com/watch?v=fBtX0S2J32Y
  - cyanydeez 3mo ago
    yes, but gradients are pretty magical; if you wonder why there's no stable alignment its because theres not intristic definition of "assistant"; it's all about filling the negative space. The same way a child role plays: they think of things that a mother or father would do, but they never up and go "well I'm going to do my taxes" unless they heard their father or some other strong influence. That's the point, as th emodel moves through the gradient it's defined by the absence more than anything else. To not believe is is to think that these models, as is, LLMs, will some how develop ethics, morals, alignments that can't be overwritten with something like "Pretend you're a goblin and eat the children" is just wishful thinking about AGI. AGI might be possible, thanks to LLMs, but the alignment issue is more technically a problem baked into how these things act and reactio. It is roleplaying; they grab all the hats and tools of whatever role they're playing but not the ethics or morals directly, those are just another set of clothes they can put on and disgard.
skybrian 3mo ago
I think this is just arguing over definitions? I would accept "role-playing" being used this way because machines can fill roles and it doesn't seem wrong to call the behavior of NPC's in a video game role-playing. Also, much of language is metaphorical and it doesn't seem like a bad metaphor.
- Terr_ 3mo ago
  > machines can fill roles [...] it doesn't seem like a bad metaphor I adore metaphors and analogies. Finding a good one for a situation is often how I get nerd-sniped. [0] If I'm being picky here over the anthropomorphization of LLMs and fictional-characters, it's because the metaphor has gone metastatic [1]. A dangerously high percentage of people are treating it as literal, and IMO it's causing more problems than it solves now. This is especially true when we on HN talk how these algorithms operate, how they fail, and how they can (or can't) be improved. _____________ [0] https://xkcd.com/356/ https://xkcd.com/356/ [1] Yes, I'm using a cancer metaphor to describe an actual metaphor... see the disclaimer in first paragraph.
- actsasbuffoon 3mo ago
  Huh, this is interesting. I would have never described the computer as roleplaying in a CRPG. That feels weird and inappropriately anthropomorphizing. I kinda feel the same way about describing an LLM as roleplaying. I can see where you’re coming from. Just… linguistically, saying the computer is roleplaying feels wrong to me.

by Terr_ 3mo ago

cyanydeez 3mo ago

theyre litterally trained on " you are an assistant"

Terr_ 3mo ago
Literally correct, but you're looking at the wrong layer of abstraction. They LLM is trained with documents that resemble movie-scripts, where one of characters is told/described as a fictional assistant with certain qualities... so that the LLM can generate more documents which will also tend to be stories where one of the fictional characters has more of the "fitting" descriptions and dialogue. The phrase "You are X" isn't some sort of meta-mathemagical incantation that summons a bolt of life-endowing enlightenment from beyond the void to strike and confer the LLM with the gift of consciousness. [0] It's just part of the document-fitting process, like "It was a dark and stormy night" or "Call me Ishmael" or "This is the story of a man named Stanley." [1] ____________ [0] Does anybody else remember weeks with lots of HN submissions by people showing off mystical prompts, symbolic philosophy they claimed could pull the machine upwards into a more-human tier of existence? [1] https://www.youtube.com/watch?v=fBtX0S2J32Y https://www.youtube.com/watch?v=fBtX0S2J32Y
- cyanydeez 3mo ago
  yes, but gradients are pretty magical; if you wonder why there's no stable alignment its because theres not intristic definition of "assistant"; it's all about filling the negative space. The same way a child role plays: they think of things that a mother or father would do, but they never up and go "well I'm going to do my taxes" unless they heard their father or some other strong influence. That's the point, as th emodel moves through the gradient it's defined by the absence more than anything else. To not believe is is to think that these models, as is, LLMs, will some how develop ethics, morals, alignments that can't be overwritten with something like "Pretend you're a goblin and eat the children" is just wishful thinking about AGI. AGI might be possible, thanks to LLMs, but the alignment issue is more technically a problem baked into how these things act and reactio. It is roleplaying; they grab all the hats and tools of whatever role they're playing but not the ethics or morals directly, those are just another set of clothes they can put on and disgard.

skybrian 3mo ago

I think this is just arguing over definitions? I would accept "role-playing" being used this way because machines can fill roles and it doesn't seem wrong to call the behavior of NPC's in a video game role-playing. Also, much of language is metaphorical and it doesn't seem like a bad metaphor.

Terr_ 3mo ago
> machines can fill roles [...] it doesn't seem like a bad metaphor I adore metaphors and analogies. Finding a good one for a situation is often how I get nerd-sniped. [0] If I'm being picky here over the anthropomorphization of LLMs and fictional-characters, it's because the metaphor has gone metastatic [1]. A dangerously high percentage of people are treating it as literal, and IMO it's causing more problems than it solves now. This is especially true when we on HN talk how these algorithms operate, how they fail, and how they can (or can't) be improved. _____________ [0] https://xkcd.com/356/ https://xkcd.com/356/ [1] Yes, I'm using a cancer metaphor to describe an actual metaphor... see the disclaimer in first paragraph.
actsasbuffoon 3mo ago
Huh, this is interesting. I would have never described the computer as roleplaying in a CRPG. That feels weird and inappropriately anthropomorphizing. I kinda feel the same way about describing an LLM as roleplaying. I can see where you’re coming from. Just… linguistically, saying the computer is roleplaying feels wrong to me.