3 ms·
World models will definitely incorporate LLMs or at least text decoders in their final architecture, LLMs are here to stay.
by khalic 2mo ago
World models will definitely incorporate LLMs or at least text decoders in their final architecture, LLMs are here to stay.
- Grombobulous 2mo agoMaybe this is a dumb analogy but is that kind of like saying “well EVs will still use lubricant! Cars that use oil are here to stay!”? The point being that the way we heavily rely on LLMs specifically right now is likely to cease to be the case, and that in the future they’ll be almost like an irrelevant component.
- khalic 2mo agoThere's far more shared concepts between them than you think. You should check the code of both world models (JEPA class for example) and compare to GPT. Many of the tricks stay the same, representation is still embeddings, there is a loss function, etc. The exact architecture will change, but unless there's a new discovery in that area, we've cracked the text component already. We're hitting the limits of LLMs because of the intrinsic limits of text as a medium. But the way we work with text is pretty much settled, fundamentally.
- Grombobulous 2mo agoThis is very interesting to me, and I have new things to read into, thank you.
- khalic 2mo agoanytime, have fun!