5 ms·

It isn't a misnomer at all, and comments like yours are why it is increasingly important to remind people about the linguistic foundations of these models. For

It isn't a misnomer at all, and comments like yours are why it is increasingly important to remind people about the linguistic foundations of these models. For example, no matter many books you read about riding a bike, you still need to actually get on a bike and do some practice before you can ride it. The reading can certainly help, at least in theory, but, in practice, is not necessary and may even hurt (if it makes certain processes that need to be unconscious held too strongly in consciousness, due to the linguistic model presented in the book). This is why LLMs being so strongly tied to natural language is still an important limitation (even it is clearly less limiting than most expected).

CamperBob2 8mo ago
You and I can't learn to ride a bike by reading thousands of books about cycling and Newtonian physics, but a robot driven by an LLM-like process certainly can. In practice it would make heavy use of RL, as humans do.
- wrs 8mo ago
  I have no idea why you used the word “certainly” there.
  - CamperBob2 8mo ago
    What is in the nature of bike-riding that cannot be reduced to text? You know transformers can do math, right?
    - anthuswilliams 8mo ago
      > What is in the nature of bike-riding that cannot be reduced to text? You're asking someone to answer this question in a text forum. This is not quite the gotcha you think it is. The distinction between "knowing" and "putting into language" is a rich source of epistemological debate going back to Plato and is still widely regarded to represent a particularly difficult philosophical conundrum. I don't see how you can make this claim with so much certainty.
      - CamperBob2 8mo ago
        Every time someone has said, "Yeah, but they can't do X," they've ended up with egg on their face. Have you seen the price of eggs lately?
        throw4847285 8mo ago
        You have resorted to, "You don't want to end up being wrong, do you?" To paraphrase Asimov, this kind of fallacious appeal is the last resort of the in-over-the-heads
        D-Machine 8mo ago
        Lot of people in this thread being caught with their pants down. Dunno what it is about LLM and AI discourse that causes people to lie or so freely offer opinions on things they clearly have no understanding about whatsoever. AI discourse truly is a great Dunning-Kruger filter.
      - D-Machine 8mo ago
        "A human can't learn to ride a bike from a book, but an LLM could" is a take so unhinged you could only find it on HN. Riding a bike is, broadly, learning to co-ordinate your muscles in response to visual data from your surroundings and signals from your vestibular and tactile systems that give you data about your movement, orientation, speed, and control. As LLMs only output tokens that represent text, by definition they can NEVER learn to ride a bike. Even ignoring that glaring definitional issue, an LLM also can't learn to ride a bike from books written by humans to humans, because an LLM could only operate through a machine using e.g. pistons and gears to manipulate the pedals. That system would be controlled by physics and mechanisms different from humans, and not have the same sensory information, so almost no human-written information about (human) bike-riding would be useful or relevant for this machine to learn how to bike. It'd just have to do reinforcement learning with some appropriate rewards and punishments for balance, speed, and falling. And if we could embody AI in a sensory system so similar to the human sensory system that it becomes plausible text on bike-riding might actually be useful to the AI, it might also be that, for exactly the same reasons, the AI learns just as well to ride just by hopping on the thing, and that the textual content is as useless to it as it is for us. Thinking this is an obvious gotcha (or the later comment that anyone thinking otherwise is going to have egg on their face) is just embarrassing. Much more of a wordcel problem than I would have expected on HN.
- D-Machine 8mo ago
  > In practice it would make heavy use of RL, as humans do. Oh, so you mean, it would be in a harness of some sort that lets it connect to sensors that tell it things about its position, speed, balance and etc? Well, yes, but then it isn't an LLM anymore, because it has more than language to model things!
  - skeptic_ai 8mo ago
    Can’t we claim the sensor data (x=5,y=9…) is text too. Not sure if it’s great just plain text, but would be better if could understand the position internally somehow.
    - D-Machine 8mo ago
      No, we can't, words have meanings, the middle "L" in LLM refers to natural language, trying to redefine everything as "language" doesn't just mean an LLM can magically do everything. In particular, sensor data doesn't have the same semantics or structure at all as language (it is continuous data and should not be tokenized; it will be multi-channel, i.e. have multiple streams, whereas text is single-channel; outputs need to be multi-channel as well, and realtime, so it is unclear if the LLM recursive approach can work at all or is appropriate). The lack of contextuality / interdependency, both within and between these streams might even mean that e.g. self-attention is not that helpful and just computationally wasteful here. E.g. what was said thousands of tokens ago can be completely relevant and change the meaning of tokens being generated now, but any bike sensor data from more than about a second or so ago is completely irrelevant to all future needed outputs. Sure, maybe a transformer might still do well processing this data, but an LLM literally can't. It would require significant architectural changes just to be able to accept the inputs and make the outputs.
      - skeptic_ai 8mo ago
        Ok. Maybe let’s talk about Claude , Gemini , ChatGPT whatever they are. I don’t care about your definition of llm, let’s talk current edge models
throw310822 8mo ago
> no matter many books you read about riding a bike, you still need to actually get on a bike and do some practice before you can ride it This is like saying that no matter how much you know theoretically about a foreign language you still need to train your brain to talk it. It has little to do with the reality of that language or the correctness of your model of it, but rather with the need to train realtime circuits to do some work. Let me try some variations: "no matter how many books you read about ancient history, you need to have lived there before you can reasonably talk about it". "No matter how many books you have read about quantum mechanics, you need to be a particle..."
- D-Machine 8mo ago
  > It has little to do with the reality of that language or the correctness of your model of it, but rather with the need to train realtime circuits to do some work. To the contrary, this is purely speculative and almost certainly wrong, riding a bike is co-ordinating the realtime circuits in the right way, and language and a linguistic model fundamentally cannot get you there. There are plenty of other domains like this, where semantic reasoning (e.g. unquantified syllogistic reasoning) just doesn't get you anywhere useful. I gave an example from cooking later in this thread. You are falling IMO into exactly the trap of the linguistic reductionist, thinking that language is the be-all and end-all of cognition. Talk to e.g. actual mathematicians, and they will generally tell you they may broadly recruit visualization, imagined tactile and proprioceptive senses, and hard-to-vocalize "intuition". One has to claim this is all epiphenomenal, or that e.g. all unconscious thought is secretly using language, to think that all modeling is fundamentally linguistic (or more broadly, token manipulation). This is not a particularly credible or plausible claim given the ubiquity of cognition across animals or from direct human experiences, so the linguistic boundedness of LLMs is very important and relevant.
  - throw310822 8mo ago
    Funny, because riding a bicycle or speaking a language is exactly something people don't have a world model of. Ask someone to explain how riding a bicycle works, or an uneducated native speaker to explain the grammar of their language. They have no clue. "Making the right movement at the right time within a narrow boundary of conditions" is a world model, or is it just predicting the next move? > You are falling IMO into exactly the trap of the linguistic reductionist, thinking that language is the be-all and end-all of cognition. I'm not saying that at all. I am saying that any (sufficiently long, varied) coherent speech needs a world model, so if something produces coherent speech, there must be a world model behind. We can agree that the model is lacking as much as the language productions are incoherent: which is very little, these days.
aixpert 8mo ago
you are living in the past these models have been trained on image data for ages, and one interesting find was that even before that they could model aspects of the visual world astonishingly well even though not perfect just through language.
- D-Machine 8mo ago
  You've clearly never read an actual paper on the models and understand nothing about backbones, pre-training, or anything I've said in my posts in this thread. I've made claims far more specific about the directionality of information flow in Large Multimodal Models, and here you are just providing generic abstract claims far too vague to address any of that. Are you using AI for these posts?
- djhn 8mo ago
  Counterpoint: Try to use an LLM for even the most coarse of visual similarity tasks for something that’s extremely abundant in the corpus. For instance, say you are a woman with a lookalike celebrity, someone who is a very close match in hair colour, facial structure, skin tone and body proportions. You would like to browse outfits worn by other celebrities (presumably put together by professional stylists) that look exactly like her. You ask an LLM to list celebrities that look like celebrity X, to then look up outfit inspiration. No matter how long the list, no matter how detailed the prompt in the features that must be matched, no matter how many rounds you do, the results will be completely unusable, because broad language dominates more specific language in the corpus. The LLM cannot adequately model these facets, because language is in practice too imprecise, as currently used by people. To dissect just one such facet, the LLM response will list dozens of people who may share a broad category (red hair), with complete disregard to the exact shade of red, whether or not the hair is dyed and whether or not it is indeed natural hair or a wig. The number of listicles clustering these actresses together as redheads will dominate anything with more specific qualifiers, like ’strawberry blonde’ (which in general counts as red hair), ’undyed hair’ (which in fact tends to increase the proportion of dyed hair results, because that’s how linguistic vector similarity works sometimes) and ’natural’ (which again seems to translate into ’the most natural looking unnatural’, because that’s how language tends to be used).

CamperBob2 8mo ago

You and I can't learn to ride a bike by reading thousands of books about cycling and Newtonian physics, but a robot driven by an LLM-like process certainly can. In practice it would make heavy use of RL, as humans do.

wrs 8mo ago
I have no idea why you used the word “certainly” there.
- CamperBob2 8mo ago
  What is in the nature of bike-riding that cannot be reduced to text? You know transformers can do math, right?
  - anthuswilliams 8mo ago
    > What is in the nature of bike-riding that cannot be reduced to text? You're asking someone to answer this question in a text forum. This is not quite the gotcha you think it is. The distinction between "knowing" and "putting into language" is a rich source of epistemological debate going back to Plato and is still widely regarded to represent a particularly difficult philosophical conundrum. I don't see how you can make this claim with so much certainty.
    - CamperBob2 8mo ago
      Every time someone has said, "Yeah, but they can't do X," they've ended up with egg on their face. Have you seen the price of eggs lately?
      - throw4847285 8mo ago
        You have resorted to, "You don't want to end up being wrong, do you?" To paraphrase Asimov, this kind of fallacious appeal is the last resort of the in-over-the-heads
        D-Machine 8mo ago
        Lot of people in this thread being caught with their pants down. Dunno what it is about LLM and AI discourse that causes people to lie or so freely offer opinions on things they clearly have no understanding about whatsoever. AI discourse truly is a great Dunning-Kruger filter.
    - D-Machine 8mo ago
      "A human can't learn to ride a bike from a book, but an LLM could" is a take so unhinged you could only find it on HN. Riding a bike is, broadly, learning to co-ordinate your muscles in response to visual data from your surroundings and signals from your vestibular and tactile systems that give you data about your movement, orientation, speed, and control. As LLMs only output tokens that represent text, by definition they can NEVER learn to ride a bike. Even ignoring that glaring definitional issue, an LLM also can't learn to ride a bike from books written by humans to humans, because an LLM could only operate through a machine using e.g. pistons and gears to manipulate the pedals. That system would be controlled by physics and mechanisms different from humans, and not have the same sensory information, so almost no human-written information about (human) bike-riding would be useful or relevant for this machine to learn how to bike. It'd just have to do reinforcement learning with some appropriate rewards and punishments for balance, speed, and falling. And if we could embody AI in a sensory system so similar to the human sensory system that it becomes plausible text on bike-riding might actually be useful to the AI, it might also be that, for exactly the same reasons, the AI learns just as well to ride just by hopping on the thing, and that the textual content is as useless to it as it is for us. Thinking this is an obvious gotcha (or the later comment that anyone thinking otherwise is going to have egg on their face) is just embarrassing. Much more of a wordcel problem than I would have expected on HN.
D-Machine 8mo ago
> In practice it would make heavy use of RL, as humans do. Oh, so you mean, it would be in a harness of some sort that lets it connect to sensors that tell it things about its position, speed, balance and etc? Well, yes, but then it isn't an LLM anymore, because it has more than language to model things!
- skeptic_ai 8mo ago
  Can’t we claim the sensor data (x=5,y=9…) is text too. Not sure if it’s great just plain text, but would be better if could understand the position internally somehow.
  - D-Machine 8mo ago
    No, we can't, words have meanings, the middle "L" in LLM refers to natural language, trying to redefine everything as "language" doesn't just mean an LLM can magically do everything. In particular, sensor data doesn't have the same semantics or structure at all as language (it is continuous data and should not be tokenized; it will be multi-channel, i.e. have multiple streams, whereas text is single-channel; outputs need to be multi-channel as well, and realtime, so it is unclear if the LLM recursive approach can work at all or is appropriate). The lack of contextuality / interdependency, both within and between these streams might even mean that e.g. self-attention is not that helpful and just computationally wasteful here. E.g. what was said thousands of tokens ago can be completely relevant and change the meaning of tokens being generated now, but any bike sensor data from more than about a second or so ago is completely irrelevant to all future needed outputs. Sure, maybe a transformer might still do well processing this data, but an LLM literally can't. It would require significant architectural changes just to be able to accept the inputs and make the outputs.
    - skeptic_ai 8mo ago
      Ok. Maybe let’s talk about Claude , Gemini , ChatGPT whatever they are. I don’t care about your definition of llm, let’s talk current edge models

throw310822 8mo ago

> no matter many books you read about riding a bike, you still need to actually get on a bike and do some practice before you can ride it This is like saying that no matter how much you know theoretically about a foreign language you still need to train your brain to talk it. It has little to do with the reality of that language or the correctness of your model of it, but rather with the need to train realtime circuits to do some work. Let me try some variations: "no matter how many books you read about ancient history, you need to have lived there before you can reasonably talk about it". "No matter how many books you have read about quantum mechanics, you need to be a particle..."

D-Machine 8mo ago
> It has little to do with the reality of that language or the correctness of your model of it, but rather with the need to train realtime circuits to do some work. To the contrary, this is purely speculative and almost certainly wrong, riding a bike is co-ordinating the realtime circuits in the right way, and language and a linguistic model fundamentally cannot get you there. There are plenty of other domains like this, where semantic reasoning (e.g. unquantified syllogistic reasoning) just doesn't get you anywhere useful. I gave an example from cooking later in this thread. You are falling IMO into exactly the trap of the linguistic reductionist, thinking that language is the be-all and end-all of cognition. Talk to e.g. actual mathematicians, and they will generally tell you they may broadly recruit visualization, imagined tactile and proprioceptive senses, and hard-to-vocalize "intuition". One has to claim this is all epiphenomenal, or that e.g. all unconscious thought is secretly using language, to think that all modeling is fundamentally linguistic (or more broadly, token manipulation). This is not a particularly credible or plausible claim given the ubiquity of cognition across animals or from direct human experiences, so the linguistic boundedness of LLMs is very important and relevant.
- throw310822 8mo ago
  Funny, because riding a bicycle or speaking a language is exactly something people don't have a world model of. Ask someone to explain how riding a bicycle works, or an uneducated native speaker to explain the grammar of their language. They have no clue. "Making the right movement at the right time within a narrow boundary of conditions" is a world model, or is it just predicting the next move? > You are falling IMO into exactly the trap of the linguistic reductionist, thinking that language is the be-all and end-all of cognition. I'm not saying that at all. I am saying that any (sufficiently long, varied) coherent speech needs a world model, so if something produces coherent speech, there must be a world model behind. We can agree that the model is lacking as much as the language productions are incoherent: which is very little, these days.

aixpert 8mo ago

you are living in the past these models have been trained on image data for ages, and one interesting find was that even before that they could model aspects of the visual world astonishingly well even though not perfect just through language.

D-Machine 8mo ago
You've clearly never read an actual paper on the models and understand nothing about backbones, pre-training, or anything I've said in my posts in this thread. I've made claims far more specific about the directionality of information flow in Large Multimodal Models, and here you are just providing generic abstract claims far too vague to address any of that. Are you using AI for these posts?
djhn 8mo ago
Counterpoint: Try to use an LLM for even the most coarse of visual similarity tasks for something that’s extremely abundant in the corpus. For instance, say you are a woman with a lookalike celebrity, someone who is a very close match in hair colour, facial structure, skin tone and body proportions. You would like to browse outfits worn by other celebrities (presumably put together by professional stylists) that look exactly like her. You ask an LLM to list celebrities that look like celebrity X, to then look up outfit inspiration. No matter how long the list, no matter how detailed the prompt in the features that must be matched, no matter how many rounds you do, the results will be completely unusable, because broad language dominates more specific language in the corpus. The LLM cannot adequately model these facets, because language is in practice too imprecise, as currently used by people. To dissect just one such facet, the LLM response will list dozens of people who may share a broad category (red hair), with complete disregard to the exact shade of red, whether or not the hair is dyed and whether or not it is indeed natural hair or a wig. The number of listicles clustering these actresses together as redheads will dominate anything with more specific qualifiers, like ’strawberry blonde’ (which in general counts as red hair), ’undyed hair’ (which in fact tends to increase the proportion of dyed hair results, because that’s how linguistic vector similarity works sometimes) and ’natural’ (which again seems to translate into ’the most natural looking unnatural’, because that’s how language tends to be used).