4 ms·

You and I can't learn to ride a bike by reading thousands of books about cycling and Newtonian physics, but a robot driven by an LLM-like process certainly can.

You and I can't learn to ride a bike by reading thousands of books about cycling and Newtonian physics, but a robot driven by an LLM-like process certainly can. In practice it would make heavy use of RL, as humans do.

wrs 9mo ago
I have no idea why you used the word “certainly” there.
- CamperBob2 9mo ago
  What is in the nature of bike-riding that cannot be reduced to text? You know transformers can do math, right?
  - anthuswilliams 9mo ago
    > What is in the nature of bike-riding that cannot be reduced to text? You're asking someone to answer this question in a text forum. This is not quite the gotcha you think it is. The distinction between "knowing" and "putting into language" is a rich source of epistemological debate going back to Plato and is still widely regarded to represent a particularly difficult philosophical conundrum. I don't see how you can make this claim with so much certainty.
    - CamperBob2 9mo ago
      Every time someone has said, "Yeah, but they can't do X," they've ended up with egg on their face. Have you seen the price of eggs lately?
      - throw4847285 9mo ago
        You have resorted to, "You don't want to end up being wrong, do you?" To paraphrase Asimov, this kind of fallacious appeal is the last resort of the in-over-the-heads
        D-Machine 9mo ago
        Lot of people in this thread being caught with their pants down. Dunno what it is about LLM and AI discourse that causes people to lie or so freely offer opinions on things they clearly have no understanding about whatsoever. AI discourse truly is a great Dunning-Kruger filter.
    - D-Machine 9mo ago
      "A human can't learn to ride a bike from a book, but an LLM could" is a take so unhinged you could only find it on HN. Riding a bike is, broadly, learning to co-ordinate your muscles in response to visual data from your surroundings and signals from your vestibular and tactile systems that give you data about your movement, orientation, speed, and control. As LLMs only output tokens that represent text, by definition they can NEVER learn to ride a bike. Even ignoring that glaring definitional issue, an LLM also can't learn to ride a bike from books written by humans to humans, because an LLM could only operate through a machine using e.g. pistons and gears to manipulate the pedals. That system would be controlled by physics and mechanisms different from humans, and not have the same sensory information, so almost no human-written information about (human) bike-riding would be useful or relevant for this machine to learn how to bike. It'd just have to do reinforcement learning with some appropriate rewards and punishments for balance, speed, and falling. And if we could embody AI in a sensory system so similar to the human sensory system that it becomes plausible text on bike-riding might actually be useful to the AI, it might also be that, for exactly the same reasons, the AI learns just as well to ride just by hopping on the thing, and that the textual content is as useless to it as it is for us. Thinking this is an obvious gotcha (or the later comment that anyone thinking otherwise is going to have egg on their face) is just embarrassing. Much more of a wordcel problem than I would have expected on HN.
      - cindyllm 9mo ago
        [dead]
      - skeptic_ai 9mo ago
        The llm can output to a tool calling which moves the bike. Can claim the same as your brain alone can’t control the bike because can’t access muscle tool. You need to compare similar stuff
        D-Machine 9mo ago
        > The LLM can output to a tool calling which moves the bike. No, it literally cannot. See the reasons I give elsewhere here: https://news.ycombinator.com/item?id=46948266 https://news.ycombinator.com/item?id=46948266. A more general AI trained with reinforcement learning and a novel architecture could surely learn to ride a bike, but an LLM simply cannot.
D-Machine 9mo ago
> In practice it would make heavy use of RL, as humans do. Oh, so you mean, it would be in a harness of some sort that lets it connect to sensors that tell it things about its position, speed, balance and etc? Well, yes, but then it isn't an LLM anymore, because it has more than language to model things!
- skeptic_ai 9mo ago
  Can’t we claim the sensor data (x=5,y=9…) is text too. Not sure if it’s great just plain text, but would be better if could understand the position internally somehow.
  - D-Machine 9mo ago
    No, we can't, words have meanings, the middle "L" in LLM refers to natural language, trying to redefine everything as "language" doesn't just mean an LLM can magically do everything. In particular, sensor data doesn't have the same semantics or structure at all as language (it is continuous data and should not be tokenized; it will be multi-channel, i.e. have multiple streams, whereas text is single-channel; outputs need to be multi-channel as well, and realtime, so it is unclear if the LLM recursive approach can work at all or is appropriate). The lack of contextuality / interdependency, both within and between these streams might even mean that e.g. self-attention is not that helpful and just computationally wasteful here. E.g. what was said thousands of tokens ago can be completely relevant and change the meaning of tokens being generated now, but any bike sensor data from more than about a second or so ago is completely irrelevant to all future needed outputs. Sure, maybe a transformer might still do well processing this data, but an LLM literally can't. It would require significant architectural changes just to be able to accept the inputs and make the outputs.
    - skeptic_ai 9mo ago
      Ok. Maybe let’s talk about Claude , Gemini , ChatGPT whatever they are. I don’t care about your definition of llm, let’s talk current edge models

wrs 9mo ago

I have no idea why you used the word “certainly” there.

CamperBob2 9mo ago
What is in the nature of bike-riding that cannot be reduced to text? You know transformers can do math, right?
- anthuswilliams 9mo ago
  > What is in the nature of bike-riding that cannot be reduced to text? You're asking someone to answer this question in a text forum. This is not quite the gotcha you think it is. The distinction between "knowing" and "putting into language" is a rich source of epistemological debate going back to Plato and is still widely regarded to represent a particularly difficult philosophical conundrum. I don't see how you can make this claim with so much certainty.
  - CamperBob2 9mo ago
    Every time someone has said, "Yeah, but they can't do X," they've ended up with egg on their face. Have you seen the price of eggs lately?
    - throw4847285 9mo ago
      You have resorted to, "You don't want to end up being wrong, do you?" To paraphrase Asimov, this kind of fallacious appeal is the last resort of the in-over-the-heads
      - D-Machine 9mo ago
        Lot of people in this thread being caught with their pants down. Dunno what it is about LLM and AI discourse that causes people to lie or so freely offer opinions on things they clearly have no understanding about whatsoever. AI discourse truly is a great Dunning-Kruger filter.
  - D-Machine 9mo ago
    "A human can't learn to ride a bike from a book, but an LLM could" is a take so unhinged you could only find it on HN. Riding a bike is, broadly, learning to co-ordinate your muscles in response to visual data from your surroundings and signals from your vestibular and tactile systems that give you data about your movement, orientation, speed, and control. As LLMs only output tokens that represent text, by definition they can NEVER learn to ride a bike. Even ignoring that glaring definitional issue, an LLM also can't learn to ride a bike from books written by humans to humans, because an LLM could only operate through a machine using e.g. pistons and gears to manipulate the pedals. That system would be controlled by physics and mechanisms different from humans, and not have the same sensory information, so almost no human-written information about (human) bike-riding would be useful or relevant for this machine to learn how to bike. It'd just have to do reinforcement learning with some appropriate rewards and punishments for balance, speed, and falling. And if we could embody AI in a sensory system so similar to the human sensory system that it becomes plausible text on bike-riding might actually be useful to the AI, it might also be that, for exactly the same reasons, the AI learns just as well to ride just by hopping on the thing, and that the textual content is as useless to it as it is for us. Thinking this is an obvious gotcha (or the later comment that anyone thinking otherwise is going to have egg on their face) is just embarrassing. Much more of a wordcel problem than I would have expected on HN.
    - cindyllm 9mo ago
      [dead]
    - skeptic_ai 9mo ago
      The llm can output to a tool calling which moves the bike. Can claim the same as your brain alone can’t control the bike because can’t access muscle tool. You need to compare similar stuff
      - D-Machine 9mo ago
        > The LLM can output to a tool calling which moves the bike. No, it literally cannot. See the reasons I give elsewhere here: https://news.ycombinator.com/item?id=46948266 https://news.ycombinator.com/item?id=46948266. A more general AI trained with reinforcement learning and a novel architecture could surely learn to ride a bike, but an LLM simply cannot.

D-Machine 9mo ago

> In practice it would make heavy use of RL, as humans do. Oh, so you mean, it would be in a harness of some sort that lets it connect to sensors that tell it things about its position, speed, balance and etc? Well, yes, but then it isn't an LLM anymore, because it has more than language to model things!

skeptic_ai 9mo ago
Can’t we claim the sensor data (x=5,y=9…) is text too. Not sure if it’s great just plain text, but would be better if could understand the position internally somehow.
- D-Machine 9mo ago
  No, we can't, words have meanings, the middle "L" in LLM refers to natural language, trying to redefine everything as "language" doesn't just mean an LLM can magically do everything. In particular, sensor data doesn't have the same semantics or structure at all as language (it is continuous data and should not be tokenized; it will be multi-channel, i.e. have multiple streams, whereas text is single-channel; outputs need to be multi-channel as well, and realtime, so it is unclear if the LLM recursive approach can work at all or is appropriate). The lack of contextuality / interdependency, both within and between these streams might even mean that e.g. self-attention is not that helpful and just computationally wasteful here. E.g. what was said thousands of tokens ago can be completely relevant and change the meaning of tokens being generated now, but any bike sensor data from more than about a second or so ago is completely irrelevant to all future needed outputs. Sure, maybe a transformer might still do well processing this data, but an LLM literally can't. It would require significant architectural changes just to be able to accept the inputs and make the outputs.
  - skeptic_ai 9mo ago
    Ok. Maybe let’s talk about Claude , Gemini , ChatGPT whatever they are. I don’t care about your definition of llm, let’s talk current edge models