4 ms·

Is my understanding wrong that LLMs are trained to emulate observed human behavior in their training data? From that follows that LLMs fit to produce all kinds

Is my understanding wrong that LLMs are trained to emulate observed human behavior in their training data? From that follows that LLMs fit to produce all kinds of human biases. Like preferring the first choice out of many, and the last our of many (primacy biases). Funnily the LLM might replicate the biases slightly wrong and by doing so produce new derived biases.

mplewis 2y ago
LLMs don't emulate human behavior. They spit out chunks of words in an order that parrots some of their training data.
- educasean 2y ago
  Is this just pedantry or is there some insight to be gleaned by the distinction you made?
  - MyOutfitIsVague 2y ago
    I can only assume that either they are trying to point out that words aren't behavior, and mimicking human writing isn't the same thing as mimicking human behavior, or it's some pot-shot at the capabilities of LLMs.
  - Xelynega 2y ago
    It's not really pedantic when there's an entire wikipedia page on the tendency for people to conflate the two: https://en.wikipedia.org/wiki/ELIZA_effect https://en.wikipedia.org/wiki/ELIZA_effect I believe the distinction they're trying to make is between "sounding like a human"(being able to create output that we understand as language) and "thinking like a human"(having the capacity for experience, empathy, semantic comprehension, etc.)
    - tavavex 2y ago
      But nowhere in the original post was there a mention of "thinking like a human". The poster said that these systems, at their core, emulate human behaviors. Writing is a human behavior - as are all the things that are requirements for writing (operating within the rules of given language/s, making logical choices, etc). The things you listed as evidence of human thinking were never implied when talking about replicating what human writing is.
- dkenyser 2y ago
  Correct me if I'm wrong, but I feel like we're splitting hairs here. > spits out chunks of words in an order that parrots some of their training data. So, if the data was created by humans then how is that different from "emulating human behavior?" Genuinely curious as this is my rough interpretation as well.
  - deleted 2y ago
    [deleted]
  - Terr_ 2y ago
    I think a common issue in LLM discussions is a confusion between author and character. Much of this confusion is deliberately encouraged by those companies in how they designed their systems. The real-world LLM takes documents and make them longer, while we humans are busy anthropomorphizing the fictional characters that appear in those documents. Our normal tendency to fake-believe in characters from books is turbocharged when it's an interactive story, and we start to think that the choose-your-own adventure character exists somewhere on the other side of the screen. > how is that different from "emulating human behavior?" Suppose I created a program that generated stories with a Klingon character, and all the real-humans agree it gives impressive output, with cohesive dialogue, understandable motivations, references to in-universe lore, etc. It wouldn't be entirely wrong to say that the program has "emulated a Klingon", but it isn't quite right either: Can you emulate something that doesn't exist in the real world? It may be better to say that my program has emulated a particular kind of output which we would normally get from a Star Trek writer.
  - mplewis 2y ago
    Humans don't write text in a stochastic manner. We have an idea, and we find words to compose to illustrate that idea. An LLM has a stream of tokens, and it picks a next token based on the last stream. If you ask an LLM a yes/no question and demand an explanation, it doesn't start with the logical reasoning. It starts with "yes, because" or "no, because" and then it comes up with a "yes" or "no" reason to go with the tokens it spit out.
    - Terr_ 2y ago
- sumeno 2y ago
  This is a more pedantic and meme-y way of saying the same thing.
- icelancer 2y ago
  Are you sure that humans are much more than this in terms of spoken/written language?
ziaowang 2y ago
This understanding is incomplete in my opinion. LLMs are more than emulating observed behavior. In the pre-training phase tasks like masked language model indeed train the model to mimic what they read (which of course contains lots of bias); but in the RLHF phase, the model tries to generate the best response judged by human evaluations (who tries to eliminate as much bias as possible in the process). In other words, they are trained to meet human expectations in this later phase. But human expectations are also not bias-free (e.g. from the preferring-the-first-choice phenomenon)
- Xelynega 2y ago
  I don't understand what you are saying. How can the RLHF phase eliminate bias if it uses a process(human input) that has the same biases as the pre-training(human input)?
  - ziaowang 2y ago
    Texts in the wild used during pre-training contain lots of biases, such as racial and sexual biases, which are picked-up by the model. During RLHF, the human evaluators are aware of such biases and are instructed to down-vote the model responses that incorporate such biases.
nthingtohide 2y ago
Not only that if future AI distrusts humanity it is because history, literature and fiction is full of such scenarios and AI will learn those patterns and associated emotions from those texts. Humanity together will be responsible for creating a monster (if that scenario happens).
- rawandriddled 2y ago
  >Humanity together Together? It would be, 1. AI programmers, 2. AI techbros and a distant 3. AI fiction/history/literature. Foo who never used the internet: not responsible. Bar who posted pictures on Facebook: not responsible. Baz who wrote machine learning, limited dataset algorithms (webmd): not responsible. Etc.
Terr_ 2y ago
I'd say it's closer to emulating human documents. In most cases, The LLM itself is a name-less and ego-less clockwork Document-Maker-Bigger. It is being run against a hidden theater-play script. The "AI assistant" (of whatever brand-name) is a fictional character seeded into the script, and the human unwittingly provides lines for a "User" character to "speak". Fresh lines for the other character are parsed and "acted out" by conventional computer code. That character is "helpful and kind and patient" in much the same way way that another character named Dracula is a "devious bloodsucker". Even when form is really good, it isn't quite the same as substance. The author/character difference may seem subtle, but I believe it's important: We are not training LLMs to be people we like, we are training them to emit text describing characters and lines that we like. It also helps in understanding prompt injection and "hallucinations", which are both much closer to mandatory features than bugs.

mplewis 2y ago

LLMs don't emulate human behavior. They spit out chunks of words in an order that parrots some of their training data.

educasean 2y ago
Is this just pedantry or is there some insight to be gleaned by the distinction you made?
- MyOutfitIsVague 2y ago
  I can only assume that either they are trying to point out that words aren't behavior, and mimicking human writing isn't the same thing as mimicking human behavior, or it's some pot-shot at the capabilities of LLMs.
- Xelynega 2y ago
  It's not really pedantic when there's an entire wikipedia page on the tendency for people to conflate the two: https://en.wikipedia.org/wiki/ELIZA_effect https://en.wikipedia.org/wiki/ELIZA_effect I believe the distinction they're trying to make is between "sounding like a human"(being able to create output that we understand as language) and "thinking like a human"(having the capacity for experience, empathy, semantic comprehension, etc.)
  - tavavex 2y ago
    But nowhere in the original post was there a mention of "thinking like a human". The poster said that these systems, at their core, emulate human behaviors. Writing is a human behavior - as are all the things that are requirements for writing (operating within the rules of given language/s, making logical choices, etc). The things you listed as evidence of human thinking were never implied when talking about replicating what human writing is.
dkenyser 2y ago
Correct me if I'm wrong, but I feel like we're splitting hairs here. > spits out chunks of words in an order that parrots some of their training data. So, if the data was created by humans then how is that different from "emulating human behavior?" Genuinely curious as this is my rough interpretation as well.
- deleted 2y ago
  [deleted]
- Terr_ 2y ago
  I think a common issue in LLM discussions is a confusion between author and character. Much of this confusion is deliberately encouraged by those companies in how they designed their systems. The real-world LLM takes documents and make them longer, while we humans are busy anthropomorphizing the fictional characters that appear in those documents. Our normal tendency to fake-believe in characters from books is turbocharged when it's an interactive story, and we start to think that the choose-your-own adventure character exists somewhere on the other side of the screen. > how is that different from "emulating human behavior?" Suppose I created a program that generated stories with a Klingon character, and all the real-humans agree it gives impressive output, with cohesive dialogue, understandable motivations, references to in-universe lore, etc. It wouldn't be entirely wrong to say that the program has "emulated a Klingon", but it isn't quite right either: Can you emulate something that doesn't exist in the real world? It may be better to say that my program has emulated a particular kind of output which we would normally get from a Star Trek writer.
- mplewis 2y ago
  Humans don't write text in a stochastic manner. We have an idea, and we find words to compose to illustrate that idea. An LLM has a stream of tokens, and it picks a next token based on the last stream. If you ask an LLM a yes/no question and demand an explanation, it doesn't start with the logical reasoning. It starts with "yes, because" or "no, because" and then it comes up with a "yes" or "no" reason to go with the tokens it spit out.
  - Terr_ 2y ago
sumeno 2y ago
This is a more pedantic and meme-y way of saying the same thing.
icelancer 2y ago
Are you sure that humans are much more than this in terms of spoken/written language?

ziaowang 2y ago

This understanding is incomplete in my opinion. LLMs are more than emulating observed behavior. In the pre-training phase tasks like masked language model indeed train the model to mimic what they read (which of course contains lots of bias); but in the RLHF phase, the model tries to generate the best response judged by human evaluations (who tries to eliminate as much bias as possible in the process). In other words, they are trained to meet human expectations in this later phase. But human expectations are also not bias-free (e.g. from the preferring-the-first-choice phenomenon)

Xelynega 2y ago
I don't understand what you are saying. How can the RLHF phase eliminate bias if it uses a process(human input) that has the same biases as the pre-training(human input)?
- ziaowang 2y ago
  Texts in the wild used during pre-training contain lots of biases, such as racial and sexual biases, which are picked-up by the model. During RLHF, the human evaluators are aware of such biases and are instructed to down-vote the model responses that incorporate such biases.

nthingtohide 2y ago

Not only that if future AI distrusts humanity it is because history, literature and fiction is full of such scenarios and AI will learn those patterns and associated emotions from those texts. Humanity together will be responsible for creating a monster (if that scenario happens).

rawandriddled 2y ago
>Humanity together Together? It would be, 1. AI programmers, 2. AI techbros and a distant 3. AI fiction/history/literature. Foo who never used the internet: not responsible. Bar who posted pictures on Facebook: not responsible. Baz who wrote machine learning, limited dataset algorithms (webmd): not responsible. Etc.

Terr_ 2y ago

I'd say it's closer to emulating human documents. In most cases, The LLM itself is a name-less and ego-less clockwork Document-Maker-Bigger. It is being run against a hidden theater-play script. The "AI assistant" (of whatever brand-name) is a fictional character seeded into the script, and the human unwittingly provides lines for a "User" character to "speak". Fresh lines for the other character are parsed and "acted out" by conventional computer code. That character is "helpful and kind and patient" in much the same way way that another character named Dracula is a "devious bloodsucker". Even when form is really good, it isn't quite the same as substance. The author/character difference may seem subtle, but I believe it's important: We are not training LLMs to be people we like, we are training them to emit text describing characters and lines that we like. It also helps in understanding prompt injection and "hallucinations", which are both much closer to mandatory features than bugs.