3 ms·

The basic pseudocode of an interactive LLM processor is while (true) context = readLine() while (true) nextToken = llm.generateNext

The basic pseudocode of an interactive LLM processor is while (true) context = readLine() while (true) nextToken = llm.generateNextToken(context) context = rtrim(context + nextToken, MAX_CONTEXT_LENGTH) if (nextToken == END_OF_LINE_TOKEN) break print (nextToken) That's a program that doesn't, in general, guarantee that its inner loop halts. It may never come back and prompt for more input. The decision to stop asking the GPT engine to keep predicting tokens after it reaches the end of an initial answer is entirely an implementation choice, based on looking at the token output and concluding that the LLM just started generating the start of something we'd rather have the user provide. You can just keep asking for more predictions. Forever if you so choose. There's no architectural reason why the sequence of tokens produced can't constitute an ongoing train of thought reasoning towards an answer... or towards a conclusion that it can't reason its way to an answer.

skepticATX 3y ago
Is it really a train of thought if the LLM completely loses tokens as soon as they are trimmed from the context?
- jameshart 3y ago
  Do you think humans have infinite capacity to store their trains of thought? We invented writing stuff down for a reason. Mathematical notation is precisely a tool for enabling humans to extend their train of thought beyond their immediate 'prompt context length'. Even with finite working memory, humans are generally considered capable of exhibiting intelligence.
  - skepticATX 3y ago
    I’d argue that humans have quite a large prompt context when you consider that we build efficient knowledge representations as we process input. I can read a 1000 page book and keep all of it in my “prompt context” even though I’m not memorizing every word that I read. There is no reason that this couldn’t be done with LLMs, but it’s certainly not what they’re doing today - and it puts you squarely back in the realm of old school AI.
    - jameshart 3y ago
      Sure, but that's an argument about scale not about fundamental limitations of the architecture. The OP is saying the LLM approach can't be intelligent. Not even if you add more parameters, more RAM, more scale. You're saying that an LLM might be able to do what you can do if we make the matrices bigger. That's a different claim. Of course, the real trick, and I think it's clear this is the missing step, is that LLM training can read a thousand page book and embed that knowledge into the weights. So what you want to do is take some of the prompt input, and use it to update the weights in the model (or the LORA, or the fine-tuning, or whatever), not just the context. That's not how LLMs are working today. It's not clear to me that we're so many steps away from that being something we can do though.
srcreigh 3y ago
The article isn't about output token limits, it's about input token limits.
- jameshart 3y ago
  Input token limits still don't mean that that program always halts. It could, after a very very long time, loop back to a previously visited state. That's the only non-halting outcome for any finite approximation of a Turing machine, after all. Including your brain.
  - srcreigh 3y ago
    The odds that any 2000 bit random input TM ever reaches a similar input tape/current state is 0. Just imagine the average TM which receives 2000 bits of random input in the first 2000 slots of the tape after every operation. What you’re saying is both mathematically and psychologically impossible. Almost nothing which receives random inputs every unit of time ever ends up at the exact same state as any previous state. The only counterexamples to this are algorithms which encode the random inputs to some concrete rules. I guarantee any such set of rules you can imagine are vastly less complex than any LLM let alone any human brain.

skepticATX 3y ago

Is it really a train of thought if the LLM completely loses tokens as soon as they are trimmed from the context?

jameshart 3y ago
Do you think humans have infinite capacity to store their trains of thought? We invented writing stuff down for a reason. Mathematical notation is precisely a tool for enabling humans to extend their train of thought beyond their immediate 'prompt context length'. Even with finite working memory, humans are generally considered capable of exhibiting intelligence.
- skepticATX 3y ago
  I’d argue that humans have quite a large prompt context when you consider that we build efficient knowledge representations as we process input. I can read a 1000 page book and keep all of it in my “prompt context” even though I’m not memorizing every word that I read. There is no reason that this couldn’t be done with LLMs, but it’s certainly not what they’re doing today - and it puts you squarely back in the realm of old school AI.
  - jameshart 3y ago
    Sure, but that's an argument about scale not about fundamental limitations of the architecture. The OP is saying the LLM approach can't be intelligent. Not even if you add more parameters, more RAM, more scale. You're saying that an LLM might be able to do what you can do if we make the matrices bigger. That's a different claim. Of course, the real trick, and I think it's clear this is the missing step, is that LLM training can read a thousand page book and embed that knowledge into the weights. So what you want to do is take some of the prompt input, and use it to update the weights in the model (or the LORA, or the fine-tuning, or whatever), not just the context. That's not how LLMs are working today. It's not clear to me that we're so many steps away from that being something we can do though.

srcreigh 3y ago

The article isn't about output token limits, it's about input token limits.

jameshart 3y ago
Input token limits still don't mean that that program always halts. It could, after a very very long time, loop back to a previously visited state. That's the only non-halting outcome for any finite approximation of a Turing machine, after all. Including your brain.
- srcreigh 3y ago
  The odds that any 2000 bit random input TM ever reaches a similar input tape/current state is 0. Just imagine the average TM which receives 2000 bits of random input in the first 2000 slots of the tape after every operation. What you’re saying is both mathematically and psychologically impossible. Almost nothing which receives random inputs every unit of time ever ends up at the exact same state as any previous state. The only counterexamples to this are algorithms which encode the random inputs to some concrete rules. I guarantee any such set of rules you can imagine are vastly less complex than any LLM let alone any human brain.