4 ms·

I am getting tired of hearing "next token predictor" from carbon-based facial expression predictors. You are saying this like an argument which allows somehow e

I am getting tired of hearing "next token predictor" from carbon-based facial expression predictors. You are saying this like an argument which allows somehow estimate upper bound of possible influence of these entities. And I do not see how this help to make predictions. It sounds to me like saying "but air is just molecules bobbing around". OK, that's true, but does it help calculate wing aerodynamic profile? Yup, it makes sequence of symbols. We already have seen that producing specific sequences of symbols is mindbogglingly powerful: merely DNA producers somehow has flown to the Moon! And yes I am quite aware of Chinese room analogy. Perfectly fine applies to humans as well: single neurons in my head do not understand language, yet I as a whole I would say do understand. Just like applying Chinese room to humans does not help to estimate what humans can do I do not see how it helps to estimate what LLM can do.

canelonesdeverd 4mo ago
It's telling that the most frequent attempt at a counter-argument is just thinly-veiled misanthropy.
- maxbond 4mo ago
  I read the misanthropy as ironic. They're applying the same reductionist logic to humans, not because they are misanthropic, but to illustrate that it doesn't help us understand the case we can all agree on. "Humans aren't sentient either" is definitely not the takeaway.
  - deleted 4mo ago
    [deleted]
  - darthoctopus 4mo ago
    What is definitely being argued unironically is "it doesn't matter that humans are sentient", and I would still consider that misanthropy
    - maxbond 4mo ago
      I don't see where they said that; could you give me a quote? It does not seem to me that they are addressing humans at all, except as a foil to LLMs.
    - CamperBob2 4mo ago
      The point is, we have no idea what "sentient" or "intelligent" even means. If we agreed on the definitions, the debate would have been settled long ago.
- kys11 4mo ago
  [dead]
solid_fuel 4mo ago
> I am getting tired of hearing "next token predictor" from carbon-based facial expression predictors. That's not even a clever swipe, and it's tiring seeing such a knee-jerk reaction to a completely accurate description. LLMs are next token predictors. People are not. Humans have an inner world and subjective experience. Humans learn through their experiences, not just backprop. Token predictors are lesser, they are not alive and will never be alive.
- Self-Perfection 4mo ago
  > People are not. Humans have an inner world and subjective experience. Humans learn through their experiences, not just backprop. https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie But this is complicated and takes us sideways. Let's say somehow we can determine if LLM has inner world or/and subjective experience. Will this new gathered piece of information affect your estimate of upper bounds of LLM capabilities? It does not affect my estimate.
  - didibus 4mo ago
    The Philosophical Zombie thought process is dumb, because zombies don't exist, so the entire premise depends on something that quire frankly might be impossible for the very reason it is arguing against.
    - deleted 4mo ago
      [deleted]
- cindyllm 4mo ago
  [dead]
- maxbond 4mo ago
  That is not knowledge, that is assumption. Let's assume we have infinite memory with constant time lookups. With a sufficiently large lookup table, you could exactly replicate the behavior of any person. You could encode it as a next-token predictor: you have precomputed every possible prefix and assigned it a next token. This is a Chinese room, but it is completely indistinguishable from an intelligent, sentient person. There is no experiment you can design to slip a piece of paper (a prompt) under the door to determine whether it is Bob or the lookup table clone of Bob inside the room. Does that make the lookup table conscious or alive? Undefined. It's the wrong question. Or it's not a question science can address. So we cannot dismiss on it's face the idea that next token predictors "are not and never will be alive" unless by "alive" you simply mean "biological," but that's not really what's debatable. The argument is also very brittle because they are not in fact all next token predictors. I doubt people making this argument would be willing to concede that diffusion models are more likely to be conscious than causal models (which I do not believe but is an implication of the argument). I'm not saying that they are conscious or sentient to be clear, but the reductionist argument that they are next token predictors and therefore don't have some property humans have is not an argument. That's going from A directly to Z. You need to flesh out the bit in the middle because that doesn't follow.
somenameforme 4mo ago
It poses a simple problem. Take humanity back not that long ago into the past and language didn't even exist - our expressed token base was practically 0. We went from that discovering the secrets of the atom, putting a man on the Moon, and more. If you put an LLM in that starting point, they're going to do nothing but endlessly cycle over basically nothing. If you give them an infinite amount of time and processing, that wouldn't change. This same issue simultaneously demonstrates how humans are not anything at all like token predictors. No matter how much time you spend remixing the tokens of primitive man, you don't get 'and here is how you land on the Moon' from it.
- scotty79 4mo ago
  Token is not a clump of letters. It's a multidimensional initial input vector that gets tweaked and transformed. GPT doesn't think in tokens. It just accepts them as input (although it happily accepts any other vectors in-between the vectors that represent tokens and finding best prompt for a given task not as tokens but as input vectors is a legitimate prompt optimization strategy). It also outputs vectors that are coerced into tokens for human consumption. Yes, it goes through tokens but possible internal meanings assigned to these tokens (when surrounded by other tokens) are infinite. That's how humans form caves got to where we are now. By associating new meanings with the same old sound clumps.
- OrangeMusic 4mo ago
  > If you give them an infinite amount of time and processing, that wouldn't change. Hrm I doubt it actually. Llms are capable of discovery, as recent math news showed. This means a "society" of Llms could likely have progress.
  - pona-a 4mo ago
    Only by having the LLM random walk the hypothesis space with a validator rejecting invalid ones. The reason why LLM hypotheses are any good is because it already consumed a civilization worth of knowledge. You couldn't have bootstrapped such system with nothing but a few priors/axioms and let it discover the universe.
21asdffdsa12 4mo ago
Imagine the dependency humanity has on such a technology, after 1/4 of a generation of time has passed- and all the students grew up with "I dont have to know anything"
- seanmcdirmid 4mo ago
  It’s more like: I don’t have to know the stuff people previously had to know, but I have to know a bunch of new things people didn't need to know in the past. Which is a common refrain throughout history.

canelonesdeverd 4mo ago

It's telling that the most frequent attempt at a counter-argument is just thinly-veiled misanthropy.

maxbond 4mo ago
I read the misanthropy as ironic. They're applying the same reductionist logic to humans, not because they are misanthropic, but to illustrate that it doesn't help us understand the case we can all agree on. "Humans aren't sentient either" is definitely not the takeaway.
- deleted 4mo ago
  [deleted]
- darthoctopus 4mo ago
  What is definitely being argued unironically is "it doesn't matter that humans are sentient", and I would still consider that misanthropy
  - maxbond 4mo ago
    I don't see where they said that; could you give me a quote? It does not seem to me that they are addressing humans at all, except as a foil to LLMs.
  - CamperBob2 4mo ago
    The point is, we have no idea what "sentient" or "intelligent" even means. If we agreed on the definitions, the debate would have been settled long ago.
kys11 4mo ago
[dead]

solid_fuel 4mo ago

> I am getting tired of hearing "next token predictor" from carbon-based facial expression predictors. That's not even a clever swipe, and it's tiring seeing such a knee-jerk reaction to a completely accurate description. LLMs are next token predictors. People are not. Humans have an inner world and subjective experience. Humans learn through their experiences, not just backprop. Token predictors are lesser, they are not alive and will never be alive.

Self-Perfection 4mo ago
> People are not. Humans have an inner world and subjective experience. Humans learn through their experiences, not just backprop. https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie But this is complicated and takes us sideways. Let's say somehow we can determine if LLM has inner world or/and subjective experience. Will this new gathered piece of information affect your estimate of upper bounds of LLM capabilities? It does not affect my estimate.
- didibus 4mo ago
  The Philosophical Zombie thought process is dumb, because zombies don't exist, so the entire premise depends on something that quire frankly might be impossible for the very reason it is arguing against.
  - deleted 4mo ago
    [deleted]
cindyllm 4mo ago
[dead]
maxbond 4mo ago
That is not knowledge, that is assumption. Let's assume we have infinite memory with constant time lookups. With a sufficiently large lookup table, you could exactly replicate the behavior of any person. You could encode it as a next-token predictor: you have precomputed every possible prefix and assigned it a next token. This is a Chinese room, but it is completely indistinguishable from an intelligent, sentient person. There is no experiment you can design to slip a piece of paper (a prompt) under the door to determine whether it is Bob or the lookup table clone of Bob inside the room. Does that make the lookup table conscious or alive? Undefined. It's the wrong question. Or it's not a question science can address. So we cannot dismiss on it's face the idea that next token predictors "are not and never will be alive" unless by "alive" you simply mean "biological," but that's not really what's debatable. The argument is also very brittle because they are not in fact all next token predictors. I doubt people making this argument would be willing to concede that diffusion models are more likely to be conscious than causal models (which I do not believe but is an implication of the argument). I'm not saying that they are conscious or sentient to be clear, but the reductionist argument that they are next token predictors and therefore don't have some property humans have is not an argument. That's going from A directly to Z. You need to flesh out the bit in the middle because that doesn't follow.

somenameforme 4mo ago

It poses a simple problem. Take humanity back not that long ago into the past and language didn't even exist - our expressed token base was practically 0. We went from that discovering the secrets of the atom, putting a man on the Moon, and more. If you put an LLM in that starting point, they're going to do nothing but endlessly cycle over basically nothing. If you give them an infinite amount of time and processing, that wouldn't change. This same issue simultaneously demonstrates how humans are not anything at all like token predictors. No matter how much time you spend remixing the tokens of primitive man, you don't get 'and here is how you land on the Moon' from it.

scotty79 4mo ago
Token is not a clump of letters. It's a multidimensional initial input vector that gets tweaked and transformed. GPT doesn't think in tokens. It just accepts them as input (although it happily accepts any other vectors in-between the vectors that represent tokens and finding best prompt for a given task not as tokens but as input vectors is a legitimate prompt optimization strategy). It also outputs vectors that are coerced into tokens for human consumption. Yes, it goes through tokens but possible internal meanings assigned to these tokens (when surrounded by other tokens) are infinite. That's how humans form caves got to where we are now. By associating new meanings with the same old sound clumps.
OrangeMusic 4mo ago
> If you give them an infinite amount of time and processing, that wouldn't change. Hrm I doubt it actually. Llms are capable of discovery, as recent math news showed. This means a "society" of Llms could likely have progress.
- pona-a 4mo ago
  Only by having the LLM random walk the hypothesis space with a validator rejecting invalid ones. The reason why LLM hypotheses are any good is because it already consumed a civilization worth of knowledge. You couldn't have bootstrapped such system with nothing but a few priors/axioms and let it discover the universe.

21asdffdsa12 4mo ago

Imagine the dependency humanity has on such a technology, after 1/4 of a generation of time has passed- and all the students grew up with "I dont have to know anything"

seanmcdirmid 4mo ago
It’s more like: I don’t have to know the stuff people previously had to know, but I have to know a bunch of new things people didn't need to know in the past. Which is a common refrain throughout history.