3 ms·

This kind of was on my mind recently, I was wondering, when I speak am I just spitting out the next word that makes sense or is there more to it. I think there

by tayo42 3y ago

This kind of was on my mind recently, I was wondering, when I speak am I just spitting out the next word that makes sense or is there more to it. I think there is for people, I can think before I speak, I can plan out my thought entirely before turning it into words. Each invocation of the chat bot/llm is a new set of probabilities. I can plan what my 2nd token output will be and stick to it. Llm models dont have the ability to think past the first. To me that is auto completing, just done better, unless models really do have a sense of planning ahead. But I don't think the really complex game playing ai do that either.

pixl97 3y ago
>I can think before I speak, I can plan out my thought entirely before turning it into words. In theory models can/could do the same thing. Think of the current text output of a model being those thoughts inside your head, you have an internal scratch space you work on ideas, then after you perform 'chain of thought' on them, you output to the world. What you're outputting to the world isn't your directly thoughts, but more of a calculation of what you wanted to achieve. Really for LLMs you just need to have the model put it's output to an internal buffer, read that buffer and make sure it makes sense, then output that to the end user. It's probably not fast or compute efficient to do that at this time, but at some point it likely will be.
- drwiggly 3y ago
  >Really for LLMs you just need to have the model put it's output to an internal buffer, read that buffer and make sure it makes sense, then output that to the end user. Makes sense to what. The LLM doesn't have a goal, other then to spew text that looks like it should be there.
  - stevenhuang 3y ago
    The analogy lies in the fact that, much like evolution through natural selection, deliberate intelligence/ability of organisms to comprehend reality is not the objective, but something else entirely is. For evolution, it's fitness. For LLMs, it's the next token. Yet despite that, the ability to reason emerges as a means to an end.
  - pixl97 3y ago
    To the terminal or instrumental goal of the statement it is working on. Question to LLM, "I have one hundred and eleven eggs in the store and another two hundred and twenty two are showing up in an hour, how many eggs will I have in total" Internal response "this looks like math problem that requires addition. the answer is 333. use a calculator validate 111 + 222. (send 111+222, receive 333). Tool returns 333 validing previous response" External response: "The answer is 333" This chain of logic is internally consistent, hence makes sense.
- tayo42 3y ago
  Yeah thinking about it more, it does seem possible. Like the attention masking done in transformers, I get why, but I also never understood why you wouldn't want attention to happen, at least about concepts and abstract ideas about what is coming up in the phrase before predicting a word
TerrifiedMouse 3y ago
> when I speak am I just spitting out the next word that makes sense or is there more to it. There is more to it. Specifically you are doing so to advance towards a specific goal. LLMs don't have goals. They just pick from a list of likely tokens - based on their training data - at random to generate the next token.
- brookst 3y ago
  So the externally provided metaprompt doesn’t establish a goal that generation works toward?
  - TerrifiedMouse 3y ago
    I don’t think LLMs work towards anything. It just picks from a list of likely “next tokens” at random.
    - brookst 3y ago
      “Likely” as defined by…?
      - TerrifiedMouse 3y ago
        Statistics as defined by training data. https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
nkingsy 3y ago
I think people get tricked by the forward pass mechanics into thinking a single generation is comparable to a human thought process. I think we have a llm like mechanism we can employ and lean on as heavily as we like, but we also have an executive function, like thousands of specialized instant Boolean checks, which can adjust and redirect the big talky model’s context on the fly. My hunch is it’s turtles all the way down. “Gut feelings” are hyper-optimized ASICS with tiny parameter counts, but all using the same intelligence mechanisms. Extrapolating from that hunch, we are already witnessing AGI, and in fact we’ve started at the top. I believe that current llms are actually far far superior to a human language center and current architectures are more than sufficient in terms of raw intelligence. The challenge that remains is to understand, train, integrate, and orchestrate all the different flavors of intelligence that humans wield so elegantly and naturally as to make them opaque to our own understanding.

by tayo42 3y ago

pixl97 3y ago

>I can think before I speak, I can plan out my thought entirely before turning it into words. In theory models can/could do the same thing. Think of the current text output of a model being those thoughts inside your head, you have an internal scratch space you work on ideas, then after you perform 'chain of thought' on them, you output to the world. What you're outputting to the world isn't your directly thoughts, but more of a calculation of what you wanted to achieve. Really for LLMs you just need to have the model put it's output to an internal buffer, read that buffer and make sure it makes sense, then output that to the end user. It's probably not fast or compute efficient to do that at this time, but at some point it likely will be.

drwiggly 3y ago
>Really for LLMs you just need to have the model put it's output to an internal buffer, read that buffer and make sure it makes sense, then output that to the end user. Makes sense to what. The LLM doesn't have a goal, other then to spew text that looks like it should be there.
- stevenhuang 3y ago
  The analogy lies in the fact that, much like evolution through natural selection, deliberate intelligence/ability of organisms to comprehend reality is not the objective, but something else entirely is. For evolution, it's fitness. For LLMs, it's the next token. Yet despite that, the ability to reason emerges as a means to an end.
- pixl97 3y ago
  To the terminal or instrumental goal of the statement it is working on. Question to LLM, "I have one hundred and eleven eggs in the store and another two hundred and twenty two are showing up in an hour, how many eggs will I have in total" Internal response "this looks like math problem that requires addition. the answer is 333. use a calculator validate 111 + 222. (send 111+222, receive 333). Tool returns 333 validing previous response" External response: "The answer is 333" This chain of logic is internally consistent, hence makes sense.
tayo42 3y ago
Yeah thinking about it more, it does seem possible. Like the attention masking done in transformers, I get why, but I also never understood why you wouldn't want attention to happen, at least about concepts and abstract ideas about what is coming up in the phrase before predicting a word

TerrifiedMouse 3y ago

> when I speak am I just spitting out the next word that makes sense or is there more to it. There is more to it. Specifically you are doing so to advance towards a specific goal. LLMs don't have goals. They just pick from a list of likely tokens - based on their training data - at random to generate the next token.

brookst 3y ago
So the externally provided metaprompt doesn’t establish a goal that generation works toward?
- TerrifiedMouse 3y ago
  I don’t think LLMs work towards anything. It just picks from a list of likely “next tokens” at random.
  - brookst 3y ago
    “Likely” as defined by…?
    - TerrifiedMouse 3y ago
      Statistics as defined by training data. https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

nkingsy 3y ago

I think people get tricked by the forward pass mechanics into thinking a single generation is comparable to a human thought process. I think we have a llm like mechanism we can employ and lean on as heavily as we like, but we also have an executive function, like thousands of specialized instant Boolean checks, which can adjust and redirect the big talky model’s context on the fly. My hunch is it’s turtles all the way down. “Gut feelings” are hyper-optimized ASICS with tiny parameter counts, but all using the same intelligence mechanisms. Extrapolating from that hunch, we are already witnessing AGI, and in fact we’ve started at the top. I believe that current llms are actually far far superior to a human language center and current architectures are more than sufficient in terms of raw intelligence. The challenge that remains is to understand, train, integrate, and orchestrate all the different flavors of intelligence that humans wield so elegantly and naturally as to make them opaque to our own understanding.