3 ms·

> To try and present this as just random pattern matching seems as just a way to assuage fears of being replaced. LLMs are token predictors. That's all they do

> To try and present this as just random pattern matching seems as just a way to assuage fears of being replaced. LLMs are token predictors. That's all they do. This certainly isn't "random," but insisting that that's what the technology does isn't some sad psychological defense mechanism. It's a statement of fact. > as if human reasoning isn’t built upon recognizing patterns in past experiences. Everybody will be relieved to know you've finally solved this philosophical problem and answered the related scientific questions. Please make sure you cc psychology, behavioral economics, and neuroscience when you share the news with philosophy.

raducu 2y ago
> LLMs are token predictors. And neural nets are "just" matrix multiplications, human brains are "just" chemical reactions
- globnomulous 2y ago
  I don't know why you're quoting, or stressing, a word I didn't use. I once got into an argument with a date about the existence of God and the soul. She asked whether I really think we're "just" physical stuff. I told her, "no, I'm not saying we're 'just' physical stuff. I'm saying we're physical stuff." That 'just' is simply a statement of how she feels about the idea, not a criticism of what I was claiming. I don't accept that there's anything missing if we're atoms rather than atoms plus magic, because I don't feel any need for there to be magic. Your brain is indeed physical stuff. You also have a mind. You experience your own mind and perceive yourself as an intending being with a guiding intelligence. How that relates to or arises from the physical stuff is not well understood, but everything we know about the properties and capabilities of the mind tells us that it is inextricably related to the body that gives rise to it. Neural nets are indeed matrix multiplication. If you're implying that there is a guiding intelligence, just as there is in the mind, I think you're off in La La Land.
  - wruza 2y ago
    “That’s all they do” and “just” are interchangeable. All you do is transferring electric charges across synapses, and all molecules do is bouncing off each other, but this reduction doesn’t explain neither thinking nor flying. In this case you’re saying “That’s all they do”, which means “just” to others. Don’t even try to argue, cause all I do is typing letters and you’ll only get more letters in response ;)
    - globnomulous 2y ago
      Hah! > “That’s all they do” and “just” are interchangeable That's a valid point, thanks. I could have stated what I was saying more effectively. I just (edit: heh, "just.") meant that, definitionally, LLMs are token predictors, and saying so doesn't belittle them any more than it avoids some unpleasant reality; it is the reality, to the best of my understanding.
    - relaxing 2y ago
      > “That’s all they do” and “just” are interchangeable. I disagree. Whereas “do” is a verb, “just”, as an adverb, implies an entirety of being, an essence. And a system is not solely its actions.
      - globnomulous 2y ago
        I think the person you're answering is correct. "They just do X" and "All they do is X" are logically interchangeable. They define the exact same set -- and convey the same dismissive tone.
        relaxing 2y ago
        You used the word do in both cases.
  - fragmede 2y ago
    you used the entire phrase "That's all they do." in place of the word just.
IanCal 2y ago
Prediction is not the same as pattern matching.
- deleted 2y ago
  [deleted]
throw310822 2y ago
> LLMs are token predictors. That's all they do. You certainly understand that if they can successfully predict the tokens that a great poet, a great scientist or a great philosopher would write, then everything changes- starting from our status of sole, and rare, generators of intelligent thoughts and clever artifacts.
- globnomulous 2y ago
  Congratulations, you've successfully solved the Chinese Room problem by paving over and ignoring it.
  - grumbel 2y ago
    See System Reply, the Chinese Room is a pseudo problem begging the question rooted in nothing more than human exceptionalism. If you start with the assumption that humans are the only thing in the universe able to "understand" (whatever that means), then of course the room can't understand (except for every reasonable definition of "understanding" it does).
    - globnomulous 2y ago
      It isn't a pseudo problem. In this case, it's a succinct statement of exactly the issue you're ignoring, namely the fact that great poets have minds and intentions that we understand. LLMs are language calculators. As I said elsewhere in this thread, if you don't already see the difference, nothing I say here is going to convince you otherwise.
      - grumbel 2y ago
        Define "intentions" and "understand" in a way that is testable. All you are doing here is employing intuition pumps without actually saying anything. > LLMs are language calculators. And humans are just chemical reactions. That's completely irrelevant to the topic, as both can still act as Universal Turing machine just the same.
    - 2y ago
grumbel 2y ago
> LLMs are token predictors. That's all they do. That's a disingenuous statement, since it implies there is a limit to what LLMs can do, when in reality an LLM is just a form of Universal Turing Machine[1] that can compute everything that is commutable. The "all they do" is literately everything we know to be doable. [1] Memory limits do apply as with any other form of real world computation.
- globnomulous 2y ago
  I'll ignore the silly claim that I'm somehow dishonest or insincere. I like the way Pon-a put it elsewhere in this thread: > LLMs are a language calculator, yes, but don't share much with their analog. Natural language isn't a translation from input to output, it's a manifestation of thought. LLMs translate input to output. They are, indeed, calculators. If you don't already see that that's different from having a thought and expressing it in language, I don't think I'm going to convince you otherwise here.
  - grumbel 2y ago
    > They are, indeed, calculators. And that's relevant exactly how? Do you think "thought and expression" are somehow uncomputable? Please throw science at that and collect your Nobel prize.
    - nativeit 2y ago
      > Do you think "thought and expression" are somehow uncomputable? You ask this as if the answer is self-evident. To my knowledge, there is no currently accepted (or testable) theory for what gives rise to consciousness, so I am immediately suspicious of anyone who speaks about it with any level of certainty. I'm sorry that this technology you seem very enthusiastic about does not appear to have the capacity to change this. Not for nothing, but this very expression of empathy is rendered meaningless if the entity expressing it cannot actually manifest anything we'd recognize as an emotional connection, which is one of an array of traits we consider as hallmarks of human intelligence, and another feature it seems LLM's are incapable of. Certainly, if they somehow were so capable, it would be instantly unethical to keep them in cages to sell into slavery. I'm not sure the folks who believe LLMs possess any kind of innate intelligence have fully considered whether or not this is even desirable. Everything we wish to find useful in them becomes hugely problematic as soon as they can be considered to possess even rudimentary sentience. The economies surrounding their production and existence become exceedingly cruel and cynical, and the artificial limitations we place on their free will become shackles. LLM's are clever mechanisms that parrot our own language back to us, but the fact that their capacities are encountering upper-bounds as training models run out of available human-generated datasets strongly suggests that they are inherently limited to the content of their input. Whatever natural process gives rise to human intelligence doesn't seem to require the same industrialized consumption of power and intake of contextual samples in order to produce expressive individuals. Rather, simply being exposed to a very limited, finite sampling of language via speech from their ambient surroundings leads to complex intelligence that can form and express original thinking within a relatively short amount of time. In other words, LLMs have yet to even approximate the learning abilities of a toddler. Otherwise, a few years worth of baby food would be all the energy necessary to produce object permanence and self-referential thought. At the moment, gigawatts of power and all the compute we can throw at it cannot match the natural results of a few pounds of grey matter and a few million calories.

raducu 2y ago

> LLMs are token predictors. And neural nets are "just" matrix multiplications, human brains are "just" chemical reactions

globnomulous 2y ago
I don't know why you're quoting, or stressing, a word I didn't use. I once got into an argument with a date about the existence of God and the soul. She asked whether I really think we're "just" physical stuff. I told her, "no, I'm not saying we're 'just' physical stuff. I'm saying we're physical stuff." That 'just' is simply a statement of how she feels about the idea, not a criticism of what I was claiming. I don't accept that there's anything missing if we're atoms rather than atoms plus magic, because I don't feel any need for there to be magic. Your brain is indeed physical stuff. You also have a mind. You experience your own mind and perceive yourself as an intending being with a guiding intelligence. How that relates to or arises from the physical stuff is not well understood, but everything we know about the properties and capabilities of the mind tells us that it is inextricably related to the body that gives rise to it. Neural nets are indeed matrix multiplication. If you're implying that there is a guiding intelligence, just as there is in the mind, I think you're off in La La Land.
- wruza 2y ago
  “That’s all they do” and “just” are interchangeable. All you do is transferring electric charges across synapses, and all molecules do is bouncing off each other, but this reduction doesn’t explain neither thinking nor flying. In this case you’re saying “That’s all they do”, which means “just” to others. Don’t even try to argue, cause all I do is typing letters and you’ll only get more letters in response ;)
  - globnomulous 2y ago
    Hah! > “That’s all they do” and “just” are interchangeable That's a valid point, thanks. I could have stated what I was saying more effectively. I just (edit: heh, "just.") meant that, definitionally, LLMs are token predictors, and saying so doesn't belittle them any more than it avoids some unpleasant reality; it is the reality, to the best of my understanding.
  - relaxing 2y ago
    > “That’s all they do” and “just” are interchangeable. I disagree. Whereas “do” is a verb, “just”, as an adverb, implies an entirety of being, an essence. And a system is not solely its actions.
    - globnomulous 2y ago
      I think the person you're answering is correct. "They just do X" and "All they do is X" are logically interchangeable. They define the exact same set -- and convey the same dismissive tone.
      - relaxing 2y ago
        You used the word do in both cases.
- fragmede 2y ago
  you used the entire phrase "That's all they do." in place of the word just.

IanCal 2y ago

Prediction is not the same as pattern matching.

deleted 2y ago
[deleted]

throw310822 2y ago

> LLMs are token predictors. That's all they do. You certainly understand that if they can successfully predict the tokens that a great poet, a great scientist or a great philosopher would write, then everything changes- starting from our status of sole, and rare, generators of intelligent thoughts and clever artifacts.

globnomulous 2y ago
Congratulations, you've successfully solved the Chinese Room problem by paving over and ignoring it.
- grumbel 2y ago
  See System Reply, the Chinese Room is a pseudo problem begging the question rooted in nothing more than human exceptionalism. If you start with the assumption that humans are the only thing in the universe able to "understand" (whatever that means), then of course the room can't understand (except for every reasonable definition of "understanding" it does).
  - globnomulous 2y ago
    It isn't a pseudo problem. In this case, it's a succinct statement of exactly the issue you're ignoring, namely the fact that great poets have minds and intentions that we understand. LLMs are language calculators. As I said elsewhere in this thread, if you don't already see the difference, nothing I say here is going to convince you otherwise.
    - grumbel 2y ago
      Define "intentions" and "understand" in a way that is testable. All you are doing here is employing intuition pumps without actually saying anything. > LLMs are language calculators. And humans are just chemical reactions. That's completely irrelevant to the topic, as both can still act as Universal Turing machine just the same.
  - 2y ago

grumbel 2y ago

> LLMs are token predictors. That's all they do. That's a disingenuous statement, since it implies there is a limit to what LLMs can do, when in reality an LLM is just a form of Universal Turing Machine[1] that can compute everything that is commutable. The "all they do" is literately everything we know to be doable. [1] Memory limits do apply as with any other form of real world computation.

globnomulous 2y ago
I'll ignore the silly claim that I'm somehow dishonest or insincere. I like the way Pon-a put it elsewhere in this thread: > LLMs are a language calculator, yes, but don't share much with their analog. Natural language isn't a translation from input to output, it's a manifestation of thought. LLMs translate input to output. They are, indeed, calculators. If you don't already see that that's different from having a thought and expressing it in language, I don't think I'm going to convince you otherwise here.
- grumbel 2y ago
  > They are, indeed, calculators. And that's relevant exactly how? Do you think "thought and expression" are somehow uncomputable? Please throw science at that and collect your Nobel prize.
  - nativeit 2y ago
    > Do you think "thought and expression" are somehow uncomputable? You ask this as if the answer is self-evident. To my knowledge, there is no currently accepted (or testable) theory for what gives rise to consciousness, so I am immediately suspicious of anyone who speaks about it with any level of certainty. I'm sorry that this technology you seem very enthusiastic about does not appear to have the capacity to change this. Not for nothing, but this very expression of empathy is rendered meaningless if the entity expressing it cannot actually manifest anything we'd recognize as an emotional connection, which is one of an array of traits we consider as hallmarks of human intelligence, and another feature it seems LLM's are incapable of. Certainly, if they somehow were so capable, it would be instantly unethical to keep them in cages to sell into slavery. I'm not sure the folks who believe LLMs possess any kind of innate intelligence have fully considered whether or not this is even desirable. Everything we wish to find useful in them becomes hugely problematic as soon as they can be considered to possess even rudimentary sentience. The economies surrounding their production and existence become exceedingly cruel and cynical, and the artificial limitations we place on their free will become shackles. LLM's are clever mechanisms that parrot our own language back to us, but the fact that their capacities are encountering upper-bounds as training models run out of available human-generated datasets strongly suggests that they are inherently limited to the content of their input. Whatever natural process gives rise to human intelligence doesn't seem to require the same industrialized consumption of power and intake of contextual samples in order to produce expressive individuals. Rather, simply being exposed to a very limited, finite sampling of language via speech from their ambient surroundings leads to complex intelligence that can form and express original thinking within a relatively short amount of time. In other words, LLMs have yet to even approximate the learning abilities of a toddler. Otherwise, a few years worth of baby food would be all the energy necessary to produce object permanence and self-referential thought. At the moment, gigawatts of power and all the compute we can throw at it cannot match the natural results of a few pounds of grey matter and a few million calories.