6 ms·

> here’s how LLMs actually work But how is that useful in any way? For all we know, LLMs are black boxes. We really have no idea how did ability to have a con

> here’s how LLMs actually work But how is that useful in any way? For all we know, LLMs are black boxes. We really have no idea how did ability to have a conversation emerge from predicting the next token.

MarkusQ 8mo ago
> We really have no idea how did ability to have a conversation emerge from predicting the next token. Uh yes, we do. It works in precisely the same way that you can walk from "here" to "there" by taking a step towards "there", and then repeating. The cognitive dissonance comes when we conflate this way of "having a conversation" (two people converse) and assume that the fact that they produce similar outputs means that they must be "doing the same thing" and it's hard to see how LLMs could be doing this. Sometimes things seems unbelievable simply because they aren't true.
- bheadmaster 8mo ago
  > It works in precisely the same way that you can walk from "here" to "there" by taking a step towards "there", and then repeating. It's funny how, in order to explain one complex phenomenon, you took an even more complex phenomenon as if it somehow simplifies it.
  - MarkusQ 8mo ago
    Sorry, can't tell if that's sarcasm or not. I wasn't referring to the biomechanical process of walking, I was referring to the process of gradient descent, which is well understood and yes, quite simple.
    - bheadmaster 8mo ago
      If that was true, knowing how elementary particles work would give us understanding of the whole universe, in which case no other science would exist. But other sciences do exist, ergo, you're wrong.
      - MarkusQ 8mo ago
        That's an interesting perspective. Thing is, that would be a rebuttal if I'd said something about the underlying tensor fiddling code being understood, and you were claiming that next token prediction was a mysterious emergent phenomena. Unfortunately that's not the argument I made. My claim is that there's nothing surprising or mysterious about the fact that a system designed to repeatedly generate a highly likely continuation of a sequence of tokens (considered as a member of pre-specified class of sequences) winds up producing something that looks like it could be a member of that class. That's kind of the whole point. These things were designed to predict plausible next tokens, and that's exactly what they do.
OkayPhysicist 8mo ago
> We really have no idea how did ability to have a conversation emerge from predicting the next token. Maybe you don't. To be clear, this is benefiting massively from hindsight, just as how if I didn't know that combustion engines worked, I probably wouldn't have dreamed up how to make one, but the emergent conversational capabilities from LLMs are pretty obvious. In a massive dataset of human writing, the answer to a question is by far the most common thing to follow a question. A normal conversational reply is the most common thing to follow a conversation opener. While impressive, these things aren't magic.
- famouswaffles 8mo ago
  >In a massive dataset of human writing, the answer to a question is by far the most common thing to follow a question. A normal conversational reply is the most common thing to follow a conversation opener. While impressive, these things aren't magic. Obviously, that's the objective, but who's to say you'll reach a goal just because you set it ? And more importantly, who's the say you have any idea how the goal has actually been achieved ? You don't need to think LLMs are magic to understand we have very little idea of what is going on inside the box.
  - measurablefunc 8mo ago
    We know exactly what is going on inside the box. The problem isn't knowing what is going on inside the box, the problem is that it's all binary arithmetic & no human being evolved to make sense of binary arithmetic so it seems like magic to you when in reality it's nothing more than a circuit w/ billions of logic gates.
    - famouswaffles 8mo ago
      We do not know or understand even a tiny fraction of the algorithms and processes a Large Language Model employs to answer any given question. We simply don't. Ironically, only the people who understand things the least think we do. Your comment about 'binary arithmetic' and 'billions of logic gates' is just nonsense.
tim333 8mo ago
I thought the Hinton talking to Jon Stewart interview gives a rough idea how they work. Hinton got Turing and Nobel prizes for inventing some of the stuff https://youtu.be/jrK3PsD3APk?t=255 https://youtu.be/jrK3PsD3APk?t=255

MarkusQ 8mo ago

> We really have no idea how did ability to have a conversation emerge from predicting the next token. Uh yes, we do. It works in precisely the same way that you can walk from "here" to "there" by taking a step towards "there", and then repeating. The cognitive dissonance comes when we conflate this way of "having a conversation" (two people converse) and assume that the fact that they produce similar outputs means that they must be "doing the same thing" and it's hard to see how LLMs could be doing this. Sometimes things seems unbelievable simply because they aren't true.

bheadmaster 8mo ago
> It works in precisely the same way that you can walk from "here" to "there" by taking a step towards "there", and then repeating. It's funny how, in order to explain one complex phenomenon, you took an even more complex phenomenon as if it somehow simplifies it.
- MarkusQ 8mo ago
  Sorry, can't tell if that's sarcasm or not. I wasn't referring to the biomechanical process of walking, I was referring to the process of gradient descent, which is well understood and yes, quite simple.
  - bheadmaster 8mo ago
    If that was true, knowing how elementary particles work would give us understanding of the whole universe, in which case no other science would exist. But other sciences do exist, ergo, you're wrong.
    - MarkusQ 8mo ago
      That's an interesting perspective. Thing is, that would be a rebuttal if I'd said something about the underlying tensor fiddling code being understood, and you were claiming that next token prediction was a mysterious emergent phenomena. Unfortunately that's not the argument I made. My claim is that there's nothing surprising or mysterious about the fact that a system designed to repeatedly generate a highly likely continuation of a sequence of tokens (considered as a member of pre-specified class of sequences) winds up producing something that looks like it could be a member of that class. That's kind of the whole point. These things were designed to predict plausible next tokens, and that's exactly what they do.

OkayPhysicist 8mo ago

> We really have no idea how did ability to have a conversation emerge from predicting the next token. Maybe you don't. To be clear, this is benefiting massively from hindsight, just as how if I didn't know that combustion engines worked, I probably wouldn't have dreamed up how to make one, but the emergent conversational capabilities from LLMs are pretty obvious. In a massive dataset of human writing, the answer to a question is by far the most common thing to follow a question. A normal conversational reply is the most common thing to follow a conversation opener. While impressive, these things aren't magic.

famouswaffles 8mo ago
>In a massive dataset of human writing, the answer to a question is by far the most common thing to follow a question. A normal conversational reply is the most common thing to follow a conversation opener. While impressive, these things aren't magic. Obviously, that's the objective, but who's to say you'll reach a goal just because you set it ? And more importantly, who's the say you have any idea how the goal has actually been achieved ? You don't need to think LLMs are magic to understand we have very little idea of what is going on inside the box.
- measurablefunc 8mo ago
  We know exactly what is going on inside the box. The problem isn't knowing what is going on inside the box, the problem is that it's all binary arithmetic & no human being evolved to make sense of binary arithmetic so it seems like magic to you when in reality it's nothing more than a circuit w/ billions of logic gates.
  - famouswaffles 8mo ago
    We do not know or understand even a tiny fraction of the algorithms and processes a Large Language Model employs to answer any given question. We simply don't. Ironically, only the people who understand things the least think we do. Your comment about 'binary arithmetic' and 'billions of logic gates' is just nonsense.

tim333 8mo ago

I thought the Hinton talking to Jon Stewart interview gives a rough idea how they work. Hinton got Turing and Nobel prizes for inventing some of the stuff https://youtu.be/jrK3PsD3APk?t=255 https://youtu.be/jrK3PsD3APk?t=255