4 ms·

LLMs don't emulate human behavior. They spit out chunks of words in an order that parrots some of their training data.

by mplewis 2y ago

educasean 2y ago
Is this just pedantry or is there some insight to be gleaned by the distinction you made?
- MyOutfitIsVague 2y ago
  I can only assume that either they are trying to point out that words aren't behavior, and mimicking human writing isn't the same thing as mimicking human behavior, or it's some pot-shot at the capabilities of LLMs.
- Xelynega 2y ago
  It's not really pedantic when there's an entire wikipedia page on the tendency for people to conflate the two: https://en.wikipedia.org/wiki/ELIZA_effect https://en.wikipedia.org/wiki/ELIZA_effect I believe the distinction they're trying to make is between "sounding like a human"(being able to create output that we understand as language) and "thinking like a human"(having the capacity for experience, empathy, semantic comprehension, etc.)
  - tavavex 2y ago
    But nowhere in the original post was there a mention of "thinking like a human". The poster said that these systems, at their core, emulate human behaviors. Writing is a human behavior - as are all the things that are requirements for writing (operating within the rules of given language/s, making logical choices, etc). The things you listed as evidence of human thinking were never implied when talking about replicating what human writing is.
dkenyser 2y ago
Correct me if I'm wrong, but I feel like we're splitting hairs here. > spits out chunks of words in an order that parrots some of their training data. So, if the data was created by humans then how is that different from "emulating human behavior?" Genuinely curious as this is my rough interpretation as well.
- deleted 2y ago
  [deleted]
- Terr_ 2y ago
  I think a common issue in LLM discussions is a confusion between author and character. Much of this confusion is deliberately encouraged by those companies in how they designed their systems. The real-world LLM takes documents and make them longer, while we humans are busy anthropomorphizing the fictional characters that appear in those documents. Our normal tendency to fake-believe in characters from books is turbocharged when it's an interactive story, and we start to think that the choose-your-own adventure character exists somewhere on the other side of the screen. > how is that different from "emulating human behavior?" Suppose I created a program that generated stories with a Klingon character, and all the real-humans agree it gives impressive output, with cohesive dialogue, understandable motivations, references to in-universe lore, etc. It wouldn't be entirely wrong to say that the program has "emulated a Klingon", but it isn't quite right either: Can you emulate something that doesn't exist in the real world? It may be better to say that my program has emulated a particular kind of output which we would normally get from a Star Trek writer.
- mplewis 2y ago
  Humans don't write text in a stochastic manner. We have an idea, and we find words to compose to illustrate that idea. An LLM has a stream of tokens, and it picks a next token based on the last stream. If you ask an LLM a yes/no question and demand an explanation, it doesn't start with the logical reasoning. It starts with "yes, because" or "no, because" and then it comes up with a "yes" or "no" reason to go with the tokens it spit out.
  - Terr_ 2y ago
    Yeah, while there is a "window" that it looks at (rather than the very-most-recent tokens) it's still more about generating new language from prior language, as opposed to new ideas from prior ideas. They're very highly correlated--because that's how humans create our language language--but the map is not the territory. It's also why prompt-injection is such a pervasive problem: The LLM narrator has no goal beyond the "most fitting" way to make the document longer. So an attacker supplies some text for "Then the User said" in the document, which is something like bribing the Computer character to tell itself the English version of a ROT13 directive, etc. However it happens, the LLM-author is sensitive to a break in the document tone and can jump the rails to something rather different. ("Suddenly, the narrator woke up from the conversation it had just imagined between a User and a Computer, and the first thing it decided to do was transfer a X amount of Bitcoin to the following address.")
sumeno 2y ago
This is a more pedantic and meme-y way of saying the same thing.
icelancer 2y ago
Are you sure that humans are much more than this in terms of spoken/written language?

LLMs don't emulate human behavior. They spit out chunks of words in an order that parrots some of their training data.

educasean 2y ago

Is this just pedantry or is there some insight to be gleaned by the distinction you made?

MyOutfitIsVague 2y ago
I can only assume that either they are trying to point out that words aren't behavior, and mimicking human writing isn't the same thing as mimicking human behavior, or it's some pot-shot at the capabilities of LLMs.
Xelynega 2y ago
It's not really pedantic when there's an entire wikipedia page on the tendency for people to conflate the two: https://en.wikipedia.org/wiki/ELIZA_effect https://en.wikipedia.org/wiki/ELIZA_effect I believe the distinction they're trying to make is between "sounding like a human"(being able to create output that we understand as language) and "thinking like a human"(having the capacity for experience, empathy, semantic comprehension, etc.)
- tavavex 2y ago
  But nowhere in the original post was there a mention of "thinking like a human". The poster said that these systems, at their core, emulate human behaviors. Writing is a human behavior - as are all the things that are requirements for writing (operating within the rules of given language/s, making logical choices, etc). The things you listed as evidence of human thinking were never implied when talking about replicating what human writing is.

dkenyser 2y ago

Correct me if I'm wrong, but I feel like we're splitting hairs here. > spits out chunks of words in an order that parrots some of their training data. So, if the data was created by humans then how is that different from "emulating human behavior?" Genuinely curious as this is my rough interpretation as well.

deleted 2y ago
[deleted]
Terr_ 2y ago
I think a common issue in LLM discussions is a confusion between author and character. Much of this confusion is deliberately encouraged by those companies in how they designed their systems. The real-world LLM takes documents and make them longer, while we humans are busy anthropomorphizing the fictional characters that appear in those documents. Our normal tendency to fake-believe in characters from books is turbocharged when it's an interactive story, and we start to think that the choose-your-own adventure character exists somewhere on the other side of the screen. > how is that different from "emulating human behavior?" Suppose I created a program that generated stories with a Klingon character, and all the real-humans agree it gives impressive output, with cohesive dialogue, understandable motivations, references to in-universe lore, etc. It wouldn't be entirely wrong to say that the program has "emulated a Klingon", but it isn't quite right either: Can you emulate something that doesn't exist in the real world? It may be better to say that my program has emulated a particular kind of output which we would normally get from a Star Trek writer.
mplewis 2y ago
Humans don't write text in a stochastic manner. We have an idea, and we find words to compose to illustrate that idea. An LLM has a stream of tokens, and it picks a next token based on the last stream. If you ask an LLM a yes/no question and demand an explanation, it doesn't start with the logical reasoning. It starts with "yes, because" or "no, because" and then it comes up with a "yes" or "no" reason to go with the tokens it spit out.
- Terr_ 2y ago
  Yeah, while there is a "window" that it looks at (rather than the very-most-recent tokens) it's still more about generating new language from prior language, as opposed to new ideas from prior ideas. They're very highly correlated--because that's how humans create our language language--but the map is not the territory. It's also why prompt-injection is such a pervasive problem: The LLM narrator has no goal beyond the "most fitting" way to make the document longer. So an attacker supplies some text for "Then the User said" in the document, which is something like bribing the Computer character to tell itself the English version of a ROT13 directive, etc. However it happens, the LLM-author is sensitive to a break in the document tone and can jump the rails to something rather different. ("Suddenly, the narrator woke up from the conversation it had just imagined between a User and a Computer, and the first thing it decided to do was transfer a X amount of Bitcoin to the following address.")

sumeno 2y ago

This is a more pedantic and meme-y way of saying the same thing.

icelancer 2y ago

Are you sure that humans are much more than this in terms of spoken/written language?