5 ms·

To use a bad analogy, it's like reading a series of articles about the cognition of a rock. It's obvious to you that rocks don't think. It's a rock. but there a

To use a bad analogy, it's like reading a series of articles about the cognition of a rock. It's obvious to you that rocks don't think. It's a rock. but there are all these articles about what the rock is thinking about, whether it's thinking, whether it had an internal monologue, what it "wants" In the same way if you understand the implementation of these types of language models you know that they simply cannot have inner monologues in the same way we do...the very concept of an inner monologue implies that there is some sort of state that we the user cannot see. Whether LLMs "think" or "want" or what thinking or wanting means etc can all be debated but whether the models have state that we cannot see is known to us: no. They do not.

sime2009 3y ago
In defense of the author, the second line of that article was: "As I said before, the insights are not particularly original, I’m just raising awareness of issues." If you are familiar in the subject matter, then that may have been your cue to stop reading.
- Zetobal 3y ago
  Raising awareness of what issue? Does he want transformers to have "inner monolog" your comment is even more confusing than the article.
famouswaffles 3y ago
Yeah well there could still be a state before generation and in response to input. Give GPT-4 about 10 or a dozen words from a meaningful sentence scrambled. Now tell it to unscramble those words to make a meaningful sentence. As long as you don't try too many tokens, the rate of meaningful completions is nearly 100%. The chances of that happening via random chance is to put it lightly, extremely small. The only way for humans to solve this kind of problem is to think about the words before the output or in this case generation.
- dizhn 3y ago
  Why would you need chance? It's applying the same parameters/algorithms to the same problem. As far as I understand you have to prompt it to provide more novel answers to the questions and there's some randomness built into the model so it doesn't sound like a broken record. This is is how it's supposed to work. Why do you consider this evidence of state?
  - famouswaffles 3y ago
    The prompt would basically be (but with less words. Around 10 is the best for consistency from my experiments. Much higher than that and it's hit or miss). Rearrange (if necessary) the following words to form a sensible sentence. Don’t modify the words, or use other words. The words are: access capabilities doesn’t done exploring general GPT-4 have have in interesting its it’s of public really researchers see since terms the to to what A successful completion would be. Since the general public doesn't have access to GPT-4, it's really interesting to see what researchers have done in terms of exploring its capabilities The number of permutations of the 24 words in the pre-scrambled sentence without taking into consideration duplicate words is 24 * 23 * 22 * ... * 3 * 2 * 1 = ~ 6.2e+23 = ~ 620,000,000,000,000,000,000,000. Taking into account duplicate words involves dividing that number by (2 * 2) = 4. It's possible that there are other permutations of those 24 words that are sensible sentences. For a language model to consistently make sensible predictions, it quite simply has to be able to "look ahead". When the probabilities for the candidate tokens for the first generated token were calculated, it seems likely that GPT-4 had calculated an internal representation of the entire sensible sentence, and elevated the probability of the first token based on that internal representation. I don't know what else to call looking backward, looking forward and then producing output anything other than a state. If you want to see what this kind of completion would look like without much or any regard for a sensible completion of the sentence then just ask 3.5.
    - californical 3y ago
      It doesn’t need to look ahead, all of the words are already in the prompt behind the current position, so it just needs to use the existing context to infer the next word Also on my second try, I gave it a slightly invalid sentence. The prompt was.. Unscramble the following: waffles eat with like cheese you. It responded with: You like to eat waffles with cheese. But notice that the prompt doesn’t have the word “to” anywhere in it. But it was the most likely word that was supposed to go there.
atemerev 3y ago
Can be simulated quite easily though. Just prompt it periodically to generate some text based on previous records, store it in some sort of long-term storage (even a text file), then sample from it and mix with each new prompt.
taneq 3y ago
Seems to me that there’s only a very short jump between having a monologue at all, and having an inner monologue. Small children go through a similar stage where they repeat speech back with increasing fidelity, then after several million years of subjective listener time they finally learn to “use their inside voice.” (Don’t worry, I’m not equating the two… but this pathway does work.)
heavyset_go 3y ago
I wish there was a way to walk back from the AI label in order to avoid sensationalism and pop sci-ification of LLMs or any model, really. Too bad that it's incredible for marketing.
thaumasiotes 3y ago
> In the same way if you understand the implementation of these types of language models you know that they simply cannot have inner monologues in the same way we do...the very concept of an inner monologue implies that there is some sort of state that we the user cannot see. I'm with you on the point you'd like to make, but LLMs obviously do have state that we cannot see. In fact, there are two different levels of obvious state: one allows the LLM to (for example) respond to input instead of ignoring it, which we may characterize as "unlearned" behavior. The level people are more interested in is the "finished model", the set of associations it learns in training. It's state and we can't see it. The internal state of the model, at those levels, doesn't change when you interact with it. But the state is there. (There is also a "working state", the state the model must keep in order to output different tokens in sequence, and in order to at some point decide that it's finished responding to a prompt. This does change during your interaction with the model, but the change is not persistent.)
logicallee 3y ago
A rock that talks in grammatical English sentences and, while while doing so, follows directions fairly well, such as following instructions that ask it to state its decision process (i.e. after you ask it to do so it modifies its behavior by doing so.) A rock that judges use to consult novel cases that aren't in the case law for the rock's opinion. A rock that passes first year medical school and law school exams. A rock programmers use to teach them things they want to learn. A rock programmers use to write small functions to their slecifications. A rock people use to write letters. A rock that can invent things if you ask it to. A rock that has a neural network for its architecture. I guess that is some rock! What made you think your analogy was bad?
- corndoge 3y ago
  Because it's prone to easily being made fun of by people such as yourself who, rather than make a good faith effort to understand what the other party was trying to communicate, choose to intentionally take a ridiculous interpretation because it allows them to lampoon what was written. I was not trying to say chatgpt is actually like a rock. But you knew that already - you just chose to ignore it in order to get the epic dunk.

sime2009 3y ago

In defense of the author, the second line of that article was: "As I said before, the insights are not particularly original, I’m just raising awareness of issues." If you are familiar in the subject matter, then that may have been your cue to stop reading.

Zetobal 3y ago
Raising awareness of what issue? Does he want transformers to have "inner monolog" your comment is even more confusing than the article.

famouswaffles 3y ago

Yeah well there could still be a state before generation and in response to input. Give GPT-4 about 10 or a dozen words from a meaningful sentence scrambled. Now tell it to unscramble those words to make a meaningful sentence. As long as you don't try too many tokens, the rate of meaningful completions is nearly 100%. The chances of that happening via random chance is to put it lightly, extremely small. The only way for humans to solve this kind of problem is to think about the words before the output or in this case generation.

dizhn 3y ago
Why would you need chance? It's applying the same parameters/algorithms to the same problem. As far as I understand you have to prompt it to provide more novel answers to the questions and there's some randomness built into the model so it doesn't sound like a broken record. This is is how it's supposed to work. Why do you consider this evidence of state?
- famouswaffles 3y ago
  The prompt would basically be (but with less words. Around 10 is the best for consistency from my experiments. Much higher than that and it's hit or miss). Rearrange (if necessary) the following words to form a sensible sentence. Don’t modify the words, or use other words. The words are: access capabilities doesn’t done exploring general GPT-4 have have in interesting its it’s of public really researchers see since terms the to to what A successful completion would be. Since the general public doesn't have access to GPT-4, it's really interesting to see what researchers have done in terms of exploring its capabilities The number of permutations of the 24 words in the pre-scrambled sentence without taking into consideration duplicate words is 24 * 23 * 22 * ... * 3 * 2 * 1 = ~ 6.2e+23 = ~ 620,000,000,000,000,000,000,000. Taking into account duplicate words involves dividing that number by (2 * 2) = 4. It's possible that there are other permutations of those 24 words that are sensible sentences. For a language model to consistently make sensible predictions, it quite simply has to be able to "look ahead". When the probabilities for the candidate tokens for the first generated token were calculated, it seems likely that GPT-4 had calculated an internal representation of the entire sensible sentence, and elevated the probability of the first token based on that internal representation. I don't know what else to call looking backward, looking forward and then producing output anything other than a state. If you want to see what this kind of completion would look like without much or any regard for a sensible completion of the sentence then just ask 3.5.
  - californical 3y ago
    It doesn’t need to look ahead, all of the words are already in the prompt behind the current position, so it just needs to use the existing context to infer the next word Also on my second try, I gave it a slightly invalid sentence. The prompt was.. Unscramble the following: waffles eat with like cheese you. It responded with: You like to eat waffles with cheese. But notice that the prompt doesn’t have the word “to” anywhere in it. But it was the most likely word that was supposed to go there.

atemerev 3y ago

Can be simulated quite easily though. Just prompt it periodically to generate some text based on previous records, store it in some sort of long-term storage (even a text file), then sample from it and mix with each new prompt.

taneq 3y ago

Seems to me that there’s only a very short jump between having a monologue at all, and having an inner monologue. Small children go through a similar stage where they repeat speech back with increasing fidelity, then after several million years of subjective listener time they finally learn to “use their inside voice.” (Don’t worry, I’m not equating the two… but this pathway does work.)

heavyset_go 3y ago

I wish there was a way to walk back from the AI label in order to avoid sensationalism and pop sci-ification of LLMs or any model, really. Too bad that it's incredible for marketing.

thaumasiotes 3y ago

> In the same way if you understand the implementation of these types of language models you know that they simply cannot have inner monologues in the same way we do...the very concept of an inner monologue implies that there is some sort of state that we the user cannot see. I'm with you on the point you'd like to make, but LLMs obviously do have state that we cannot see. In fact, there are two different levels of obvious state: one allows the LLM to (for example) respond to input instead of ignoring it, which we may characterize as "unlearned" behavior. The level people are more interested in is the "finished model", the set of associations it learns in training. It's state and we can't see it. The internal state of the model, at those levels, doesn't change when you interact with it. But the state is there. (There is also a "working state", the state the model must keep in order to output different tokens in sequence, and in order to at some point decide that it's finished responding to a prompt. This does change during your interaction with the model, but the change is not persistent.)

logicallee 3y ago

A rock that talks in grammatical English sentences and, while while doing so, follows directions fairly well, such as following instructions that ask it to state its decision process (i.e. after you ask it to do so it modifies its behavior by doing so.) A rock that judges use to consult novel cases that aren't in the case law for the rock's opinion. A rock that passes first year medical school and law school exams. A rock programmers use to teach them things they want to learn. A rock programmers use to write small functions to their slecifications. A rock people use to write letters. A rock that can invent things if you ask it to. A rock that has a neural network for its architecture. I guess that is some rock! What made you think your analogy was bad?

corndoge 3y ago
Because it's prone to easily being made fun of by people such as yourself who, rather than make a good faith effort to understand what the other party was trying to communicate, choose to intentionally take a ridiculous interpretation because it allows them to lampoon what was written. I was not trying to say chatgpt is actually like a rock. But you knew that already - you just chose to ignore it in order to get the epic dunk.