3 ms·

Why is it disheartening? LLMs are a dead-end technology with respect to AI.

khalic 2mo ago
Have you been living under a rock? We might not get "super-intelligences" (if such a thing even exists) from LLMs but they've proven incredibly useful in basically anything relating to text, including code.
- Grombobulous 2mo ago
  The point is that they’re going to be surpassed by a much better technology that doesn’t have the inherent and unavoidable downsides of LLMs. Like, right now especially in the US you’d probably sound crazy saying that one day nobody’s going to buy a gasoline car, and that EVs are going to completely replace them. But that’s basically an inevitability based on the direction we know technology is going to go, it’s only a matter of when.
  - khalic 2mo ago
    World models will definitely incorporate LLMs or at least text decoders in their final architecture, LLMs are here to stay.
    - Grombobulous 2mo ago
      Maybe this is a dumb analogy but is that kind of like saying “well EVs will still use lubricant! Cars that use oil are here to stay!”? The point being that the way we heavily rely on LLMs specifically right now is likely to cease to be the case, and that in the future they’ll be almost like an irrelevant component.
      - khalic 2mo ago
        There's far more shared concepts between them than you think. You should check the code of both world models (JEPA class for example) and compare to GPT. Many of the tricks stay the same, representation is still embeddings, there is a loss function, etc. The exact architecture will change, but unless there's a new discovery in that area, we've cracked the text component already. We're hitting the limits of LLMs because of the intrinsic limits of text as a medium. But the way we work with text is pretty much settled, fundamentally.
        Grombobulous 1mo ago
        This is very interesting to me, and I have new things to read into, thank you.
        khalic 1mo ago
        anytime, have fun!

Have you been living under a rock? We might not get "super-intelligences" (if such a thing even exists) from LLMs but they've proven incredibly useful in basically anything relating to text, including code.

Grombobulous 2mo ago
The point is that they’re going to be surpassed by a much better technology that doesn’t have the inherent and unavoidable downsides of LLMs. Like, right now especially in the US you’d probably sound crazy saying that one day nobody’s going to buy a gasoline car, and that EVs are going to completely replace them. But that’s basically an inevitability based on the direction we know technology is going to go, it’s only a matter of when.
- khalic 2mo ago
  World models will definitely incorporate LLMs or at least text decoders in their final architecture, LLMs are here to stay.
  - Grombobulous 2mo ago
    Maybe this is a dumb analogy but is that kind of like saying “well EVs will still use lubricant! Cars that use oil are here to stay!”? The point being that the way we heavily rely on LLMs specifically right now is likely to cease to be the case, and that in the future they’ll be almost like an irrelevant component.
    - khalic 2mo ago
      There's far more shared concepts between them than you think. You should check the code of both world models (JEPA class for example) and compare to GPT. Many of the tricks stay the same, representation is still embeddings, there is a loss function, etc. The exact architecture will change, but unless there's a new discovery in that area, we've cracked the text component already. We're hitting the limits of LLMs because of the intrinsic limits of text as a medium. But the way we work with text is pretty much settled, fundamentally.
      - Grombobulous 1mo ago
        This is very interesting to me, and I have new things to read into, thank you.
        khalic 1mo ago
        anytime, have fun!